Air Pollution: Limitations of EPA's Motor Vehicle Emissions Model and Plans to Address Them
Released: 1997-09-30T00:00:00

Letter

Background

The current MOBILE model, MOBILE5a, also known as the EPA mobile source emissions factor model, is a computer program that estimates the emissions of carbon monoxide, hydrocarbons, and nitrogen oxides for eight different types of gasoline-fueled and diesel highway motor vehicles. The model consists of an integrated collection of mathematical equations and assumptions about the emissions from vehicles manufactured from 1960 to 2020; generally, the cars produced in the 25 most recent model years are assumed to be in operation in any given calendar year. The first MOBILE model was made available for use in 1978; since that time, major updates and improvements to the model have been made as more has become known about the complexity of the factors affecting vehicle emissions, as measurement devices have improved, and as more data have been collected. According to agency officials, these improvements have resulted in the refinement of emissions estimates for evaporative emissions (such as occur when the fuel tank and fuel system heat up on a hot summer day); for the uncorrected in-use deterioration (wear and tear) that results from poor vehicle maintenance or tampering; and for other factors.

In its simplest form, EPA’s MOBILE model allows the model user to produce a number—an estimated quantity of emissions for the three pollutants of concern—by multiplying the estimated emissions per mile for an average urban trip times the estimated number of trip miles traveled in an area. Over the years, however, researchers have learned that vehicle emissions are highly complex. For example, EPA and others have indications today that as much as half of all hydrocarbon emissions from motor vehicles are evaporative emissions, under certain conditions. To compensate for the complexities of these and other emissions-producing activities, EPA has periodically adjusted its basic formula—through the use of revised “correction factors”—to approximate vehicle exhaust emissions in a range of situations. In essence, the correction factor is a multiplier added to the basic formula (miles traveled times emissions rate per mile) to adjust the model’s output to more closely reflect actual emissions. Except for California, EPA supplies the baseline emissions rates and correction factors for other model users—primarily state and local agencies—that typically supply their own estimates of the number of vehicle miles traveled, according to agency officials, as well as many other local area parameters, such as the average ambient temperature, vehicle classifications, and types of fuels sold.

The MOBILE model exists because precise information about the emissions behavior of the approximately 200 million vehicles in use in the United States is not known, yet the need exists to estimate the impact of motor vehicles on air quality. For the states, the MOBILE model is a tool for constructing emissions inventories, creating control strategies, producing state implementation plans (SIP), and—subsequently— demonstrating control strategy effectiveness to EPA and others. For example, the states are allowed to vary a number of control strategy features, including the types of fuels used, the type of inspection and maintenance (I&M) testing network, the frequency of I&M testing, the ages and types of vehicles to be inspected, the stringency of the tailpipe test, the number and percent of inspected vehicles that may receive a waiver,and a host of other factors. The states may choose among a number of control options as long as the state’s control strategy achieves at least as many reductions as required by the Clean Air Act.

For EPA, the MOBILE model is a tool for evaluating the adequacy of a state’s emissions inventory estimate, motor vehicle control strategies, and implementation plans. In essence, the model’s estimates provide EPA regulators with critical information that is used to evaluate the adequacy of a state’s program and the relative benefits of various policies to control motor vehicle emissions. Additionally, the model’s estimates can affect state policy decisions on issues such as the content and volatility of fuels, and some decisions on highway improvement projects. For example, the 1991 Intermodal Surface Transportation Efficiency Act (ISTEA) required, among other things, that state transportation improvement programs in certain nonattainment areas conform with the applicable state implementation plan developed under the Clean Air Act. Although the model’s original purpose was to support the development of mobile source emissions inventories, over the years its role and influence have been expanded considerably. Today, its estimates have a substantial influence not only on state and local programs but also on the automobile and oil industries, environmental and trade organizations, the public, and others. According to estimates derived partly from the MOBILE model, motor vehicles produce about 90 percent of the carbon monoxide, 50 percent of the hydrocarbons, and 30 percent of the nitrogen oxides emitted annually in major urban areas.

Limitations in the Current Model

EPA officials are examining 14 areas in the current MOBILE model in which major limitations exist. According to agency officials, it is their plan for each new version of the MOBILE model to reflect the most recent testing, data collection, and research that are available. They pointed out that EPA has updated the estimating capabilities of its MOBILE source emissions model 10 times since the model was first introduced in 1978. Table 1 briefly summarizes the areas in which major limitations exist, as well as EPA’s plans to address these limitations in its next revision to the model, MOBILE6, due to be issued in late 1998. (Additional information on these 14 areas is provided in app. I.)

While acknowledging that some vehicle emissions-producing activities are not accounted for in the current model and that other emissions-producing activities are not adequately represented in the current model on the basis of the most recent information, EPA officials said that it is important to note that EPA has conducted and/or partially funded some of the studies that have led to the new data that now question the old estimates and assumptions. Additionally, they said that EPA has work under way to address most of these limitations. For example, since its formation in September 1995, an EPA-sponsored Federal Advisory Committee Act (FACA) subcommittee workgroup has identified 47 high-priority items for improvement in the current model, and EPA and workgroup representatives are examining these limitations. EPA is also in the process of developing new procedures for improving models in general, which are discussed below.

EPA’s Process for Improving the MOBILE Model

Several model experts told us that it is the nature of models such as MOBILE5a to have limitations and to be in a continuous improvement mode. Agency officials agreed with this assessment, noting that the current model is better than any previous versions and reflects consistent growth in the quality and quantity of information available on very complex issues. Additionally, they pointed out that—through the FACA workgroup process—the revisions to the next MOBILE model, MOBILE6, have been undertaken with significantly increased openness and input from other government agencies, academia, the automobile and oil industries, environmental groups, and others. Our contacts with representatives of these groups confirmed this increased level of external stakeholders’ involvement in preparation for MOBILE6. Several commended EPA’s efforts in recent years to reach out to persons outside of the agency, and some noted that the outreach effort had given them a much greater appreciation of the model.

While acknowledging that, historically, there have been few firm criteria on the processes that should be followed when creating or revising a model such as the MOBILE model, the executive director of EPA’s Science Advisory Board (SAB) told us that the agency has a project under way to develop agencywide procedures for improving models. According to the project director, the Office of Research and Development (ORD) is planning a workshop in December 1997 to discuss the status of modeling across several media and other modeling issues, including the need for better agencywide modeling procedures. The project director also said that the Science Advisory Board’s January 1989 resolution on models was one of the best documents available on the processes for creating and improving models in general. The SAB executive director and the ORD project director told us that in their opinion, there are specific actions—most of which were recommended by the SAB in its 1989 resolution—that, when followed, can enhance a model’s predictive capabilities. Among other things, these actions include the following:

Obtaining external stakeholders’ input to ensure that the model’s assumptions and formulas receive critical review by those not involved in the model’s development.

Documenting the implicit and explicit assumptions so that others can evaluate the basis of the formulas embedded in the model.

Performing sensitivity analyses over key parameters to identify the most sensitive parameters and to establish the areas most in need of further research.

Verifying the adequacy of the model’s mathematical code.

Testing the model’s predictions with laboratory and field data to confirm that the model generates results consistent with its underlying theory.

Conducting peer review to enhance the quality, credibility, and acceptability of the model’s applications.

The SAB executive director pointed out that because of continuing concerns with the quasi-regulatory use of agency models, EPA issued agencywide guidance in 1994 specifically calling for the peer review of such models. This directive was a follow-on to EPA’s January 1993 agencywide policy requiring peer review of the scientific and technical work products used to support agency decisions. In September 1996 and March 1997, we reported and testified on the uneven implementation of EPA’s peer review policy, including that the MOBILE model had not been peer reviewed. EPA agreed with our recommendations for educating staff and managers about the merits of and procedures for conducting peer review and for ensuring that all relevant products are considered for peer review. The agency has set in motion a three-pronged approach to improve the implementation of peer review agencywide, including peer review of the next MOBILE model. According to the ORD project director, it is too early in EPA’s study to predict whether the agency may recommend that EPA require its offices to follow the other actions listed above when creating or revising models.

According to Office of Mobile Sources (OMS) officials, they plan to carry out all six of the above activities as part of their improvement process for MOBILE6 and noted that some of these activities are already well under way, such as involving external stakeholders. For example, OMS held its first stakeholder meeting in June 1994, established a FACA mobile modeling workgroup in July 1995, and has held five meetings since that time to obtain external views by those not involved in the model’s development, according to agency officials. OMS officials acknowledged that some of the key formulas in the current model have not been properly documented,that full-scale sensitivity analyses have not been performed since May 1990 (when they were performed for MOBILE4.1), that fewer resources have resulted in fewer confirming data, and that the MOBILE model has not been peer reviewed. However, they said they have efforts under way or planned to address these and other modeling needs. For example, one of the recommendations of the FACA mobile modeling workgroup—made up of representatives from EPA, state and local agencies, industry, environmental groups, and academia—is that EPA more fully document the model’s assumptions. Additionally, OMS plans to perform sensitivity analyses for the next version of the model and to have the studies supporting key changes for MOBILE6 peer reviewed. Also, OMS officials explained that as changes are proposed for each area of major limitations in the model, they plan to have the entire area peer reviewed.

Agency officials explained that declining modeling resources have affected the pace of model improvements over the years, particularly their ability to confirm the model’s estimates with large numbers of vehicle tests. For example, a study of the emissions characteristics of 100 passenger cars for both exhaust and evaporative emissions could cost from $1.4 to $1.6 million, according to the agency’s current estimates, and still not address the emissions impacts of road grade, air conditioning, or most fuel studies. Studies of heavy duty trucks and other larger vehicles would cost considerably more. As an illustration of the magnitude of the task compared with available resources, we obtained EPA’s estimate of mobile modeling needs in response to the mobile source requirements envisioned for the 1990 Clean Air Act amendments. This June 1990 analysis indicated that the Office of Mobile Sources would need about $60 million for the modeling improvements known at that time. Since then, because of higher priority needs, OMS has been allocated only $21.8 million, cumulatively, for modeling improvements, although the research needs have increased. However, many more groups have become involved in non-EPA-funded vehicle emissions studies than in the past, allowing EPA to benefit from their studies and observations. Additionally, in some instances, researchers have sought EPA’s input on study protocols beforehand and have shared the data collected with EPA afterwards. While concerned about resources, OMS officials explained that making model improvements is an ongoing, continuous process—and one that will continue after MOBILE6 is issued in 1998. They pointed out that their goal is for each new version of the MOBILE model to reflect the latest testing, data collection, and research. While still not able to quantify the improvements, they said that in their opinion, each new version of the MOBILE model is better than its predecessor.

Agency Comments

We provided copies of a draft of this report to the Environmental Protection Agency for its review and comment. We obtained comments from EPA officials, including the Director of the EPA Office of Mobile Sources. EPA agreed with the overall message of the report but expressed concerns with imprecise language and suggested several changes to clarify information in the report. For example, EPA suggested that in lieu of describing some limitations associated with its use of the Federal Test Procedure as a basis for estimating emissions as “FTP assumptions,” since the FTP is a specific, codified test cycle, these would be better described as “FTP parameters.” EPA also suggested that we provide specific citations for four studies referred to in appendix I. We made the language changes suggested by EPA, including adding the citations.

EPA was also concerned that the report did not clearly distinguish between limitations that may result in only trivial emissions impacts and those that could be significant. Where it was possible to do so, we believe the estimated emissions impact had already been quantified or qualitatively described in the report. Additionally, as noted in the section on the uncertainty limitation, one of the 14 limitations of the MOBILE model is that it does not currently have information about, nor estimates of, the uncertainty associated with its emissions estimates. However, we agree that researchers viewed some limitations as having a more significant impact on emissions than others, and we have provided this view in the report. Appendix II contains the agency’s overall written comments.

We conducted our review from October 1996 through August 1997 in accordance with generally accepted government auditing standards. A detailed discussion of our scope and methodology is provided in appendix III.

As arranged with your office, unless you publicly announce its contents earlier, we plan no further distribution of this report until 15 days after the date of this letter. At that time, we will send copies to the Administrator of EPA and other interested parties. We will also make copies available to others upon request.

Please call me at (202) 512-9692 if you or your staff have any questions. Major contributors to this report are listed in appendix IV.

Major Limitations in MOBILE5a

According to agency officials and model experts we contacted, it is the nature of models to have limitations and to be in a continuous improvement mode. As a result, the Environmental Protection Agency (EPA) has periodically updated the estimating capabilities of its mobile source emissions model to reflect new information as data have become available; MOBILE5a reflects the 10th major revision since the model was first introduced in 1978. The following sections provide additional information on 14 areas in which major limitations exist in EPA’s current MOBILE model, MOBILE5a.

Five Limitations Related to Use of the Federal Test Procedure as a Basis for Inventory Modeling

The underlying basis for EPA’s original model, and all subsequent versions, has been the Federal Test Procedure (FTP), a laboratory dynamometertest used to certify new cars against new-car emissions standards. The FTP is roughly based on a typical urban area trip, complete with starts and stops, covering 7.5 miles in the Los Angeles urban area in the late 1960s. Such a trip is known as a driving cycle, which can be approximated on a dynamometer. Primarily because of the limitations in past dynamometers, the FTP driving cycle parameters stipulate, among other things, that vehicles average 19.6 miles per hour (mph) over the 7.5 mile trip, do not exceed 57 mph, accelerate gradually (not to exceed 3.3 mph/second), and travel on a flat surface. Additionally, EPA added 10 percent to the FTP dynamometer load in an attempt to simulate the effects of air conditioner usage. However, five of the major limitations in the current MOBILE model relate to FTP parameters. These five are (1) emissions from road grade, (2) emissions from air conditioner usage, (3) emissions at higher speeds, (4) emissions from aggressive driving, and (5) emissions immediately after engine start-up (cold- start). Agency officials have long recognized that some of these original FTP parameters were not representative of actual driving conditions and, to compensate for these limitations, have added correction factors to the MOBILE model to estimate what emissions would be for speeds in excess of 57 mph, for rapid acceleration beyond 3.3 mph/second, and for other scenarios, such as different temperatures or different fuels. For example, the impact of temperature on emissions can be substantial. Consequently, while FTP testing has been performed between 68 and 86 degrees Fahrenheit, EPA’s MOBILE model used a correction factor to estimate that 1995 exhaust emissions of hydrocarbons would be 3 times greater at 25 degrees than at the FTP temperatures.

However, MOBILE5a does not account for the impact of road grade—such as when a car climbs a hill—although some studies have indicated that both the increased load on the engine from climbing a hill and the decreased load that accompanies engine deceleration significantly increase vehicle emissions. According to agency officials, it is not expected that MOBILE6 will have adjustments for road grade, although such adjustments are being planned for MOBILE7. In addition to being uncertain about the amount of emissions related to road grade, EPA officials explained that obtaining the basic data from instrumented cars and chase cars to make such adjustments would be expensive at this time and that because of the cost and length of time required for these studies, the impact of road grade will probably not be addressed until MOBILE7. Additionally, an equally important consideration is that once these basic data on the effects of road grade on emissions are obtained, state and local agencies would have to plot road grades for millions of miles of roadways in their jurisdictions, also a costly and time-consuming activity. However, as global positioning technologyfor vehicles becomes less costly and more widely available and used, it is envisioned that the impacts of road grade emissions will be modeled in the future. According to EPA officials, the amount of data needed to estimate the impact of road grade is still years away. They also noted that time would be needed to develop consistent guidance on how state and local agencies should go about collecting these data.

With respect to the representation of other emissions-producing activities not represented by the FTP, EPA officials have made periodic adjustments but recognize that the current model’s correction factor adjustments may not reflect the most up-to-date information. For example, as noted in the EPA-FACA materials disseminated in March 1997, the 10-percent additional load intended to simulate the effect of air conditioner usage “is obsolete.” More recent information indicates that nitrogen oxide emissions may be from 30 to 75 percent greater at some speeds than the current model’s estimates when the air conditioner is used.

Additionally, increases in speeds above 65 mph have left data gaps in the current model that are not adequately represented by existing correction factors. Others have recognized that the MOBILE model’s estimates are inextricably tied to the FTP’s parameters, and some studies have questioned the representativeness of key assumptions as they relate to the FTP. For example, the California Air Resources Board (CARB) commissioned Sierra Research in 1993 to develop an improved driving cycle—known as the Unified Driving Cycle—by using an instrumented “chase car” to better characterize typical urban driving patterns. Among other things, this driving cycle allows cars to travel up to 67.2 mph (versus 57 mph for the FTP), allows for acceleration at a rate of up to 6.9 mph/second (versus a maximum of 3.3 mph/second for the FTP), and uses an average speed of 24.6 mph (versus 19.2 for the FTP). Several model experts believe these parameters more closely approximate actual driving conditions today. According to a 1993 CARB study, the FTP may underestimate hydrocarbon, carbon monoxide, and nitrogen oxide emissions by 27, 68, and 17 percent, respectively.

A 1993 study sponsored by EPA’s FTP improvement project found that more than one-third of the trips studied had acceleration rates of more than 7 mph/second—more than double the FTP’s maximum of 3.3 mph/second rate. Similarly, another 1993 EPA-sponsored study of instrumented vehicles in the Baltimore area found that 18 percent of total driving time in the area was composed of higher speeds and sharper accelerations than those represented on the FTP test. Also, a 1995 National Research Council report noted that aggressive driving with many accelerations resulted in hydrocarbon and carbon monoxide emissions being 14 and 15 times higher, respectively, than the emissions from average driving over the same 7-mile trip. According to the 1995 National Research Council report, “Virtually all motor vehicle testing has been based on a limited set of driving test cycles that inadequately represent current urban driving conditions.” However, one model expert told us that it took more than 1 year to evaluate one component of the model and that collecting vehicle emissions data on large data sets is very costly. EPA officials pointed out that there is not a consistent definition of what constitutes aggressive driving, that aggressive driving happens only over a portion of the trip and is highly variable among drivers, and that the above observations are not representative of average driving patterns.

The Congress has also recognized that the FTP may not reflect actual driving conditions. Concerned about the gap between emissions as measured by the FTP and actual, real-world emissions, in 1990 the Congress added Section 206(h) to the Clean Air Act, which required EPA to review and revise the FTP within 18 months “to insure that vehicles are tested under circumstances which reflect the actual current driving conditions under which motor vehicles are used.” EPA’s October 1996 final rule on FTP revisions addressed four emissions-producing activities that, according to the rule’s preamble, are not adequately represented in the current FTP. These emissions-producing activities include (1) aggressive driving behavior (such as high acceleration rates and high speeds), (2) rapid speed fluctuations (such as quick deceleration), (3) emissions immediately after engine start-up, a period when—because of the fact that engines are designed to operate at higher temperatures—emissions typically bypass emissions controls for an estimated 3 to 5 minutes until the engine reaches normal operating temperatures, and (4) actual air conditioner usage.

EPA has not yet revised the MOBILE model to reflect the results of recent studies that have led to these FTP rule revisions but has work ongoing in all four areas. According to agency officials, although adjustments had been made to the MOBILE model for most of these activities prior to issuing the revised FTP rule, given the state of knowledge today, it appears that these activities may not be adequately represented in MOBILE5a. EPA officials told us that incorporating new estimates for these emissions-producing activities would be a high priority for MOBILE6, due to be issued in late 1998. For example, in March 1997 agency officials announced their plans to substantially revise the cold-start segment of the next MOBILE model, moving—for the first time ever—from an areawide, trip-based model to a roadway-specific model that also separately accounts for start-up emissions. Under this revised model, the magnitude of start-up emissions will not depend on vehicle speed or the driving cycle. Instead, EPA is proposing to allow model users to model the emissions impacts of cold starts on the basis of local areas’ estimates of the number of such starts. Additionally, model users will be able to estimate emissions for three different types of roadways—freeways, arterials, and local roadways.

Eight Limitations Related to Other Areas of the MOBILE Model

In addition to concerns about the representativeness of the FTP parameters, the following issues were also identified by model experts, workgroup participants, and/or stakeholders we contacted. In each instance, EPA officials agreed that the limitation is an area of concern and in most cases noted that the agency has ongoing work to address the issue, which is discussed below.

One concern is the representation of high emitters in EPA’s MOBILE model database, since the data now indicate that this group of vehicles accounts for a disproportionate amount of an area’s overall emissions and that if this subset of the overall vehicle population is underrepresented, the impact on the emissions estimates can be substantial. For example, Sierra Research testified in 1995 that the worst polluting 22 percent of the vehicles produce about 50 percent of the emissions, and EPA estimates that, overall, from 10 to 30 percent of the vehicles cause the bulk of the pollution problems.

As noted by the nonfederal co-chair of the FACA modeling workgroup in a 1993 study, “The general problem of failing to control for significant factors is compounded by the likelihood that a small fraction of the vehicle fleet are currently responsible for a large percentage of vehicle emissions.” He told us that he still believes this to be one of the most significant issues facing EPA today, primarily because a very small number of vehicles can potentially be responsible for unusually high levels of pollution. This was also a significant issue for one of the Session Chairs for the Coordinating Research Council’s (CRC) April 1997 Workshop, who noted that, in his opinion, this is the single greatest issue that EPA faces—how to identify and repair high emitters and properly represent them in the modeling database. The nonfederal co-chair also noted that if the occurrence of such vehicles is not properly represented in the model, the model’s emissions estimates can be seriously flawed. He and others have concerns that the existing database may underrepresent high emitters because, among other reasons, the owners of such vehicles may avoid surrendering such vehicles for inspection and maintenance (I&M) and other testing at a higher rate than the normal population. As noted in a February 1996 study, “individuals with intentionally tampered or poorly maintained vehicles may be less likely to offer their vehicles for testing.”Additionally, a 1993 CARB study of 186 vehicles indicated that high emitters could represent 16.8 percent of the California fleet, or nearly 5 times the assumption in the California model. EPA officials have some concerns with the study, its reliance on remote sensing devices, and its applicability to other states. Additionally, EPA officials believe that the larger data sets provided by their ongoing I&M lane testing in three other states properly identify most high emitters. However, they agreed that appropriate representation of high emitters is important to the model’s emissions estimates and noted that this is also a high priority issue currently being addressed by EPA and one of the subgroups of the mobile modeling workgroup.

A second concern is the current correction factors for lower volatility fuels and for oxygenated fuels. For example, a February 1997 report by Sierra Research found, among other things, that MOBILE5a likely underestimates the impact of low reid vapor pressure (RVP) fuels on hydrocarbon and carbon monoxide emissions at temperatures above 75 degrees. The report notes that the correction factor for low RVP fuels has not changed since February 1989, when limited data on fuels with RVPs lower than 9.0 pounds per square inch (psi) caused EPA to place a constraint code in the model precluding users from being able to calculate reductions below this level. The limited data collected since that time indicate that reducing fuel RVP from 9.0 psi to 7.0 psi may reduce hydrocarbon and carbon monoxide exhaust emissions from 18 to 27 percent more than the model estimates, respectively. The January 1997 Auto/Oil Air Quality Improvement Research Program (AQIRP) Final Report suggested that reducing fuel volatility by 1 psi, from 9.0 to 8.0 psi, would reduce exhaust CO by 9 percent, exhaust HC by 4 percent, and total evaporative HC by 34 percent (NOx remained unchanged). The RVPs for most fuels used to be higher than 9.0 psi, but today they can go lower than 7.0 psi. Similarly, the model currently has no emissions reduction credits for low sulfur fuels (except, according to agency officials, for the lower sulfur effect in reformulated gasoline), although recent studies suggest that lowering the concentration of sulfur in fuel reduces the emissions of hydrocarbons and nitrogen oxides.

EPA officials said that they plan to eliminate the constraint code in the next model, MOBILE6, which will allow users to receive credit for correction factors for fuels lower than 9.0 psi; however, they noted that work in this area is still ongoing and that the data on the emissions benefits of lower RVP fuels, as well as low sulfur fuels, are limited. In addition, a 1996 National Research Council study suggested that the model may overestimate the benefits of oxygenated fuels. For example, the study noted that EPA’s MOBILE model “apparently overpredicts the oxygenated fuel effect by at least a factor of two” when the model’s estimate of carbon monoxide reductions is compared with observed data. Similarly, a 1997 study of wintertime oxygenated fuels suggested that the observed oxygenated fuel benefits were much lower than the 20 to 30 percent estimated by EPA’s model. EPA officials agreed that this is also an area that needs more study, but one which they plan to address in MOBILE6.

A third concern is MOBILE5a’ estimates of emissions system deterioration for vehicles with more than 50,000 odometer miles. This concern stems from studies that have questioned the rate and quantity of the data supporting EPA’s significantly higher rate of emissions system deterioration once vehicles reach 50,000 odometer miles. For example, prior to MOBILE5, the model assumed that a vehicle with 100,000 miles emitted about 1.0 grams of hydrocarbons for each mile driven, or about 4 times the amount a new car would emit. However, EPA adjusted the deterioration rates for vehicles with more than 50,000 miles beginning with MOBILE5 (Dec. 1992) so that the MOBILE model’s deterioration formula now calculates that the same car was emitting about 2.0 grams of hydrocarbons for each mile driven, or about 8 times the amount a new car would emit.

EPA acknowledges that these adjustments were made on the basis of limited data and that only recently have 1990-technology vehicles become old enough to accurately assess their emissions deterioration.

An October 1996 Sierra Research study of 75 vehicles with over 100,000 odometer miles questioned whether EPA had perhaps adjusted the formula too much, resulting in a model that currently overestimates the emissions from vehicles with 50,000 or more odometer miles. Among other things, the study found that EPA’s current model estimated that 80 percent or more of these higher mileage vehicles (which, on average, had accumulated 123,900 odometer miles) would be high emitters, whereas the study found that only 32 percent of the vehicles fell into this category. Similarly, an April 1997 study of 227 vehicles (model years 1991 to 1993) with more than 50,000 odometer miles found no significant changes in emissions or deterioration as indicated by the current model. OMS officials said they used all the data that were available to them at the time (1991-1992) to estimate the deterioration rate of such vehicles and that researchers since then had had more time and more vehicles to test than were available to EPA. They also said the model’s correction factor for vehicles with 50,000 or more odometer miles would likely be lowered in MOBILE6, but they were uncertain at the time of our audit how much this emissions estimate would be reduced. A special subgroup of the FACA mobile model workgroup has been established to address this issue, and their work is still ongoing.

A fourth concern is MOBILE5a’ emissions credits and assumptions about inspection and maintenance programs. According to a 1995 National Research Council report, vehicle condition—whether the vehicle is well maintained, or has been tampered with or is malfunctioning—is more important than vehicle age in determining emissions. Among other issues, there is a need to update the basic data supporting I&M emissions reduction credits to reflect a growing population of vehicles in which rates of tampering may be diminishing since tampering with newer vehicles adversely affects gas mileage and vehicle performance. Also, vehicle owners often replace older, carbureted vehicles with newer fuel-injected vehicles.

Additionally, according to agency officials, the current model provides no additional I&M credits for vehicles equipped with on-board diagnostics(OBD), a requirement for all 1994 and later light duty vehicles and trucks. This vehicle computer technology alerts a car owner when an emissions system malfunctions, permitting quicker repairs than when such malfunctions are identified through an I&M testing program, and diagnostic trouble codes assist mechanics in making better repairs. Additionally, newer vehicles have up to 8 years or 80,000-mile emissions control system warranties for two components (the on-board computer and catalytic converter), which should equate to less-polluting vehicles as a result of more durable emissions control systems and the requirement that manufacturers cover the costs of certain repairs. The current model does not provide specific credits for this growing population of OBD-equipped vehicles designed and believed to have less in-use deterioration than their predecessors.

Also, more recent and more complete data are needed on the effectiveness of repairs in an I&M program, including the adequacy and durability of these repairs, actual participation rates, and impact of remote sensing efforts. Except for remote sensing, the current model’s estimates for these parameters is based on aging and limited data. For example, EPA has not performed any tampering surveys since 1992, and agency officials said that as a result of this lack of data, the tampering assumptions for MOBILE6 will remain unchanged from MOBILE5a. However, EPA’s goal for MOBILE6 is to provide users with greater flexibility in designing I&M programs, as long as the state or local programs’ estimated I&M credits can be substantiated with state or local data. With respect to newer vehicles equipped with on-board diagnostics, because of the limited data on the longer-term emissions impact of this technology, the agency has provided credit for these OBD-equipped vehicles equal to that provided for operating an enhanced I&M program. EPA officials said that, in addition to the states’ own studies, the agency currently has I&M effectiveness studies being carried out in three states, but they were unsure whether sufficient data would be available in time to further revise the I&M assumptions in MOBILE6.

A fifth concern is the proper representation of diurnal emissions. Diurnal emissions refer only to hydrocarbons and are a form of evaporative emissions that occur when a vehicle is parked and the ambient temperature is fluctuating. For all previous versions of the MOBILE model, the data supporting these 8- to 24-hour emissions estimates were collected during a 1-hour period during which temperatures were forcibly increased over a range of temperatures. More recent testing over 24-hour and longer periods without constraining temperature increases to a one-hour period indicates some differences from the MOBILE5a estimates for such evaporative emissions. Also, evaporative emissions from vehicles with fuel leaks are now believed to be so significant that, for MOBILE6, EPA plans to model these emissions separately from other evaporative emissions. According to EPA’s most recent data, indications are that some vehicles with fuel leaks—similar to super emitters of exhaust/tailpipe emissions—can exceed the evaporative emissions of corresponding vehicles by one to two orders of magnitude. A 1996 automotive industry study of 150 vehicles found that 24-hour diurnal emissions ranged from 0.6 grams of HC to 777.2 grams of HC, with vehicles with liquid fuel leaks providing the vast majority of the emissions. EPA officials explained that while they may develop a separate category for some vehicles with significant fuel leaks, this does not necessarily mean there will be a significant overall increase in evaporative emissions estimates because, until more data are collected, there is no clear indication that these emissions were significantly underestimated in prior diurnal estimates. EPA has testing under way to determine how to better define this category of vehicles with significant fuel leaks and also plans tests to estimate their distribution within the current fleet, rate of occurrence as a function of accumulated mileage, vehicle age, and/or vehicle technology, such as fuel tank design. Agency officials are uncertain at this time whether the correction factors for other evaporative emissions estimates will be revised for MOBILE6.

A sixth concern is the adequacy of the data supporting MOBILE5a’assumptions about emissions for in-use I&M heavy duty vehicles. According to EPA’s June 1994 workshop on state needs, the in-use credits for heavy duty gasoline-powered vehicles are based on data approximately 20 years old, and there has been much change in the technology and emissions rates of these vehicles since that time. Still, the certification standards are higher for heavy duty vehicles than their light duty counterparts, and they are generally older and driven more miles annually than their light duty counterparts. EPA officials said that testing heavy duty vehicles is difficult and quite expensive and agreed that there is a lack of recent data on the in-use emissions from this category of vehicles once they have been put in service. While some studies are under way, EPA does not envision at this time that significant changes in the in-use emissions rates for heavy duty vehicles will be included in MOBILE6.

A seventh concern is the fleet characterization data in EPA’s database, stemming from a concern that much of the data used for MOBILE5a are quite old. For example, MOBILE5a’ estimates are based on the assumption that, on average, light duty vehicles are driven about 14,000 miles annually when new, decreasing to less than 10,000 miles annually after 10 years. More recent data from the U.S. Department of Transportation indicates that passenger cars are driven about 2,000 miles more annually than currently estimated by EPA’s MOBILE model, or nearly a 10-percent increase over MOBILE5a. According to a 1996 report, because of the linkage between odometer mileage and I&M program assumptions, a small change in mileage accumulation rates can result in a large impact on emissions estimations. EPA officials pointed out that the agency’s guidance encourages model users to provide their own accumulated mileage estimates; thus, they said the default values for accumulated mileage in MOBILE5a would be a problem only in those cases in which model users fail to provide their own accumulated mileage estimates. Additionally, heavy duty gasoline-powered vehicles, which have higher certification standards than their light duty counterparts, are believed to comprise a significantly larger percentage of the overall vehicle fleet than currently estimated by the MOBILE model. According to OMS, the agency plans to update the fleet characterization data for MOBILE6, including reflecting the increases in the heavy-duty vehicle population.

Similarly, another fleet characterization issue involves urban buses. For example, the current model does not have a separate classification for urban buses, although this is a growing vehicle category in many urban areas with unique operating characteristics, such as very frequent starts and stops. EPA officials explained that while buses have not been a separate category in MOBILE5a, EPA plans to expand the current list of vehicle categories from 8 to 20, one of which will be a separate category for buses.

An eighth concern is the level of distinctions in roadway classifications. The MOBILE model was originally designed only for estimating areawide emissions on the basis of assumptions associated with an entire trip. It was not designed for making decisions for various roadway classifications, such as transportation improvement projects for urban interstate, rural arterial, or urban feeder/collector streets. Several model experts have pointed out that the same average travel speed—35 mph, for instance—would indicate smooth traffic flow on a local street but severe congestion on a freeway. EPA officials pointed out that MOBILE6 will allow users to separate start emissions from any linkage to the FTP driving cycle assumptions and will also provide different correction factors for speed and driving cycle for three different types of roadways—freeways, arterials, and local roadways. Additionally, while not planned for MOBILE6, the agency plans to partially fund ongoing research with the Department of Transportation to develop a modal emissions model that may one day allow users to model additional parameters, such as the relative emissions impact of sequencing traffic signals to enhance traffic flow.

As noted above, an EPA-sponsored FACA mobile model workgroup made up of representatives from other federal, state, and local government agencies, academia, the automobile and oil industries, environmental groups, and others has been assisting EPA in improving the current model, and much of the research to fill data gaps and update aging databases was still ongoing at the time of our audit. Agency officials said that it is their plan for each new version of the MOBILE model to reflect the most recent testing, data collection, and research that are available. Except for the impact of road grade on emissions and revising the in-use credits for heavy duty vehicles, EPA officials said they plan to address each of the above limitations in the next revision, MOBILE6. However, as discussed below, the agency will not be able to quantify the uncertainty associated with its MOBILE model estimates, primarily because of the complexity and timing of factors affecting vehicle emissions and the high cost of vehicle studies.

The Uncertainty Limitation

“Uncertainty is pervasive in all three emission modeling components: vehicle activity, activity-specific emission rates, and emission rate correction factors. Uncertainty is compounded in the methodologies used to develop the emission inventory. That is, vehicle activity uncertainty is combined with emission rate uncertainty that has already been combined with correction factor uncertainty.”

Additionally, the limited work in this area indicates there are significant uncertainties associated with the current MOBILE model’s estimates. For example, one study found that “the range of uncertainty is huge” for a change in one variable—average vehicle speed—of the many variables contained in the MOBILE model. According to the study, most model users generally believe that increasing average vehicle speed from 30 mph to 50 mph will reduce vehicle emissions (because of less congested driving, with more driving at cruising speeds). The study noted that EPA’s MOBILE model estimates a 24-percent reduction in carbon monoxide emissions by increasing average vehicle speed from 30 mph to 50 mph. However, when a 95-percent confidence interval is applied, the change in emissions can range from a 72-percent decrease in carbon monoxide emissions to a 75-percent increase in such emissions. Similarly, a 1996 study of EPA’s speed correction factors for vehicle exhaust emissions found substantial uncertainty in EPA’s current MOBILE model. Among other things, the study concluded that the MOBILE model may significantly underestimate carbon monoxide and hydrocarbon emissions—“by up to 3 orders of magnitude”—as the model relates to changes in vehicle speed.

According to Office of Mobile Sources officials, EPA has been unable to quantify the model’s uncertainty primarily because of the cost and time associated with such quantification, the fact that the on-road vehicle population is a constantly changing universe of differing emissions control devices and levels, technological limitations in measurement devices, and because there is substantial naturally occurring variability in vehicle emissions (leading to further data gaps/limitations). Agency officials pointed out that there is substantial variability across (1) vehicle types (such as model year, emissions control system, engine type), (2) vehicle operating conditions (cold start, load, speed), (3) the external environment (road grade, temperature, humidity, altitude), (4) vehicle fuels (reformulated, oxygenated, reid vapor pressure), and (5) driver behavior (quick starts and stops, timing and frequency of trips). For these reasons, OMS officials told us they do not plan to develop uncertainty ranges with the next revision to the MOBILE model. Similar to the global positioning issue for road grade, they said significant technological advancement may be needed before it becomes cost-effective to address this issue. For example, future vehicles may have on-board computers with the ability to instantaneously record and later report emissions under different operating scenarios. EPA officials pointed out that it is their plan, at some point in the future, to report uncertainty ranges for some model estimates. They said they are currently saving both qualitative and quantitative descriptors for the data being collected by and for the agency in order to perform these calculations in the future.

Comments From the Environmental Protection Agency

Objectives, Scope, and Methodology

The Chairman, Subcommittee on Oversight and Investigations, House Committee on Commerce, asked us to (1) describe the major limitations in the current version of EPA’s MOBILE model and (2) describe EPA’s process for improving both current and future versions of the MOBILE model.

To describe the major limitations in the current model, we obtained and reviewed the MOBILE5a User’s Guide; EPA/OMS model documentation; the most recent published sensitivity analyses (MOBILE4.1, May 1990); relevant EPA guidance and memorandums on the MOBILE model; selected vehicle studies; the results of stakeholders meetings about the model; and the charter, objectives, minutes, and proceedings of the EPA-Federal Advisory Committee Act mobile modeling workgroup. We also reviewed five electronic databases for studies pertaining to the EPA MOBILE model; attended one mobile sources symposium where modeling issues were discussed; and attended the March 1997 FACA public workshop. We also obtained and discussed studies relating to potential MOBILE model limitations with selected representatives of state and local agencies, academia, industry, environmental groups, consulting firms, and other government agencies. Additionally, we interviewed officials and obtained documents from EPA’s Office of Mobile Sources in Ann Arbor, Michigan; Office of Research and Development in Research Triangle Park, North Carolina, and Athens, Georgia; and the EPA Science Advisory Board in Washington, D.C. We also discussed model limitations with individuals identified to us by EPA or other representatives, as well as through our own efforts as noted above.

To describe EPA’s process for improving both current and future versions of the MOBILE model, we obtained and discussed information from knowledgeable EPA/OMS air quality officials relative to ongoing activities and documented plans for making model revisions. We also discussed EPA’s past, ongoing, and planned actions with representatives of academia, industry, environmental groups, consulting firms, and other government agencies and observed one process—EPA’s open solicitation of input by external stakeholders not involved in the model’s development—at work. We also obtained documents and discussed EPA’s process for improving models in general with the Science Advisory Board and EPA’s Office of Research and Development.

We conducted our review from October 1996 though August 1997 in accordance with generally accepted government auditing standards.

Major Contributors to This Report

Resources, Community, and Economic Development Division, Washington, D.C.

Lawrence J. Dyckman, Associate Director William F. McGee, Assistant Director Judy K. Pagano, Senior Operations Research Analyst James R. Beusse, Evaluator-in-Charge Hamilton C. Greene, Jr., Staff Evaluator DeAndrea M. Leach, Staff Evaluator The first copy of each GAO report and testimony is free. Additional copies are $2 each. Orders should be sent to the following address, accompanied by a check or money order made out to the Superintendent of Documents, when necessary. VISA and MasterCard credit cards are accepted, also. Orders for 100 or more copies to be mailed to a single address are discounted 25 percent.

U.S. General Accounting Office P.O. Box 37050 Washington, DC 20013 Room 1100 700 4th St. NW (corner of 4th and G Sts. NW) U.S. General Accounting Office Washington, DC Orders may also be placed by calling (202) 512-6000 or by using fax number (202) 512-6061, or TDD (202) 512-2537.

Each day, GAO issues a list of newly available reports and testimony. To receive facsimile copies of the daily list or any list from the past 30 days, please call (202) 512-6000 using a touchtone phone. A recorded menu will provide information on how to obtain these lists.