Opinion ID: 446520
Heading Depth: 3
Heading Rank: 1

Heading: Calculation of Effectiveness in Limiting Damage

Text: 71 The most satisfactory method of comparing the ability of different bumper systems to prevent or reduce damage is to examine systems that have actually been produced and used. Although there was a wealth of real-world data for comparing the Part 581 5.0 mph system with unregulated bumpers, none existed for the primary alternative--a 2.5 mph standard with uniform bumper heights. 72 NHTSA considered several methodologies for estimating the effectiveness of the 2.5 mph alternative. One was to use real-life data on the performance of MY 1973 cars equipped with 2.5 mph rear bumpers. That was rejected, primarily because the MY 1973 rear bumpers were not subject to the height-standardizing pendulum test, and thus were not comparable to the proposed 2.5 mph bumpers. 47 Fed.Reg. 21,831; FRIA at III-3 to -4. Also rejected was a method proposed in the comments of insurance companies--reliance on laboratory condition crash test results comparing the performance of three pairs of vehicles purportedly equipped with 5.0 mph and 2.5 mph bumper systems--because there was no evidence that the bumpers used in the tests had been tested or designed in accordance with the relevant bumper standards. 47 Fed.Reg. 21,83 1-32; 47 Fed.Reg. 56,647-48; FRIA at III-5 to -6. There were also questions about the generalizability of the test data. Id. at III-7 to -8. 73 The agency decided instead to use the methodology employed in its January 1979 and June 1979 assessments of the bumper standard, see Department of Transportation, NHTSA, Analysis of the Bumper Standard (Jan. 26, 1979), J.A. 1245-52 (Preliminary Assessment); Department of Transportation, NHTSA, Final Assessment of the Bumper Standard (June 1979), J.A. 979-1182 (Final Assessment), which relied on engineering estimates of the theoretical relative effectiveness of different bumper systems. The data used in that exercise were compiled by NHTSA in a convenient table form in the FRIA, VI-3, reproduced here as Table III. 74 NOTE--Some parts of this form are wider than one screen. To view 75 material that exceeds the width of this screen, use the right arrow 76 key. To return to the original screen, use the left arrow key. 77 TABLE III Estimated Reduction in Lifetime Repair Costs Per Car Assuming Unrepaired Damage Valued at 50% and 75% of Full Cost to Repair 50% Value for Unrepaired Damage --------------------------- Repaired Unrepaired Damage Reduction in Damage Lifetime Repair Costs Per Total Accident Value Percent Value Lifeti- Effectiveness Compared of of of me in to Percent of Damage Lifetime Damage Damage Reducing Unregulat- ed Speed Lifetime Per Number of Per Per Damage 4 Bumpers 3 Number (mph) of Accide- Accide- Accidents Accide- Accide- Accident 3 2.5 mph 5 mph 2.5 nts 1 nt 2 1 nt nt 2 mph ------------------------------------------------------------------------------------------------------ 0-3 14.8 $ 150 54.1 $ 75 $63 76% 85% $48 $54 3-5 12.1 225 3.8 113 32 29 68 9 21 5-7 6.6 480 0 240 32 0 40 0 13 7-10 4.4 850 0 425 37 0 6 0 2 10-15 3.6 1,750 0 875 63 0 0 0 0 15 0.5 3,700 0 1,850 20 0 0 0 0 -------- ------ Total $57 $90 2.5/5 mph System=63% (57/90) 75% Value For Unrepaired Damage --------------------------- 0-3 14.8 150 54.1 113 84 76 85 63 71 3-5 12.1 225 3.8 169 34 29 68 10 23 5-7 6.6 480 0 360 32 0 40 0 13 7-10 4.4 850 0 638 37 0 6 0 2 10-15 3.6 1,750 0 1,313 3 0 0 0 0 15 0.5 3,700 0 2,775 20 0 0 0 0 -------- ------ Total $73 $109 2.5/5 mph System=67% (73/109) 1. From page 24 of 1979 Assessment, original source is Ford data, finetuned by State Farm data.1/5 2. From page 28 of 1979 Assessment, for unregulated bumpers.1/5 3. Per accident--values need to be multiplied by the total number oflifetime damage producing accidents to attain lifetime dollar values of damage.1/5 4. From page 30 of 1979 Assessment.1/5 78 [243 U.S.App.D.C. 138] First, the agency adopted the estimates from the 1979 Final Assessment of the percentage of damage-producing accidents that occur over the lifetime of a car at each of several speed intervals (Column 1 of the Table), FRIA at VI-2, providing separate figures for accidents resulting in damage subsequently repaired (Column 2) and damage [243 U.S.App.D.C. 139] left unrepaired (Column 4). (These figures were based on Ford Motor Company studies of relative bumper effectiveness. Final Assessment at 24, J.A. 1005.) Second, for each speed interval, NHTSA estimated separately the dollar value of repaired and unrepaired damage per accident for a car equipped with unregulated (baseline) bumpers. These data, also derived from the 1979 Final Assessment, were based on an assumed relationship between repair cost and impact speed, id. at 25-26, J.A. 1006-07, and closely approximated the results of calculations based on studies relating damage to the angle of impact, id.at B-1 to -7, J.A. 1099-1105. 16 The figures are provided in Column 3 of the Table for repaired and Column 5 for unrepaired damage. Third, by multiplying the values in Columns 2 and 3, and 4 and 5, respectively, and summing the resultant products, NHTSA obtained values of the total lifetime damage per accident for each speed category (Column 6). 79 NHTSA then adopted the 1979 estimates of the relative effectiveness of the 5.0 mph and 2.5 mph bumpers as against unregulated bumpers at each speed interval. 47 Fed.Reg. 56,648-49; FRIA at VI-2. Those relative effectiveness values are defined as the percentage of damage prevented by a bumper system compared to the unregulated bumper, Final Assessment at 29, J.A. 1010, and are reported in Columns 7 and 8. By way of illustration, at a speed of 0.0-3.0 mph, a 2.5 mph bumper prevents 76 percent of the damage and a 5.0 mph bumper prevents 85 percent of the damage that would have resulted with unregulated bumpers. Finally, the agency used the effectiveness values to determine the per-accident lifetime value of all repaired and unrepaired damage prevented by the two bumper systems. FRIA at VI-5 to -10. That computation is provided in Columns 9 and 10. The result, shown on Table III under Columns 9 and 10, is that the 2.5 mph system is 63 percent as effective as 5.0 mph bumpers (assuming that the economic value of damage an owner elects not to repair is 50 percent of the estimated cost of repair) or 67 percent as effective as 5.0 mph bumpers (assuming that unrepaired damage is valued at 75 percent of the estimated repair cost) in reducing the dollar value of damage expected from unregulated bumpers. FRIA at VI-4. 80 NHTSA then applied this conclusion to updated data on per-accident costs of unregulated and 5.0 mph bumpers, 17 and the number of lifetime accidents per vehicle, 18 to derive the present value of lifetime damage prevention for different bumper systems. FRIA at VI-5 to -11. The result is a disadvantage of the 2.5 mph system relative to the 5.0 mph system of $34-$69, FRIA at VI-10. (The range is attributable to the use of ranges for the discount factor for unrepaired damage (50-75 percent of the cost of repair), the number of lifetime accidents (2.45-3.20), and the value of lifetime damage incurred by 5.0 mph bumpers ($390-$578). FRIA at VI-8 to -10.) 81 Petitioners focus their challenge not on the soundness of the above-described methodology, but on the derivation of certain of the values shown in Table III. Even if the alleged deficiencies were clearly established, we would have some difficulty in believing that they are of such significance as to render the product before us capricious. For all of them inhered in the 1979 Final Assessment and were not thought worthy of mention by many of these same petitioners who were asked for and who furnished extensive comments at that time. Indeed, their utterly invalidating effect did not impress itself upon any of the commenters in the present rulemaking, until after [243 U.S.App.D.C. 140] the final rule was issued in 1982. We have nonetheless considered each of them and find them to be without merit. 82 Amicus National Association of Independent Insurers (NAII) directs its criticism at NHTSA's computation of the frequency of accidents by speed intervals (Columns 2 and 4 of Table III). As noted above, NHTSA derived these figures from Ford Motor Company data, which in turn relied on barrier crash data for 1969-70 model year cars reported by the Insurance Institute for Highway Safety (IIHS), see Affidavit of E.J. Rohn, Addendum to Brief for Intervenor Motor Vehicle Manufacturers Association (MVMA) at 2a (Rohn Affidavit). The Ford study computed accident frequencies for speed cells of 0.0-3.0 mph, 3.0-5.0 mph, 5.0-7.0 mph, 7.0-10 mph, and over-10 mph. See Letter from E.J. Rohn to D.W. McCallum at Chart I (Sept. 19, 1974), J.A. 2413C. The agency adopted these speed cells, FRIA at VI-2, though it broke down the over-10 mph range into 10-15 mph and over-15 mph cells using State Farm data. See Final Assessment at A-4, J.A. 1076. IIHS, however, had not conducted any barrier crashes at speeds less than 5.0 mph, leading NAII to charge here that Ford had no basis for breaking down the crucial 0.0-5.0 mph cell (containing 84% of all accidents in the Ford study) into 0.0-3.0 mph and 3.0-5.0 mph subcells, and that NHTSA therefore acted arbitrarily in relying on that information in constructing its effectiveness methodology. 83 This challenge is unfounded. The engineers who conducted the Ford study had collected data on the costs of repaired and unrepaired collision damage to about 18,000 cars, which established a distribution of accidents by damage. In order to translate this distribution into a distribution of accidents by speed, they relied upon IIHS test data that established average damage values of $208 at 5.0 mph, $597 at 10 mph and $853 at 15 mph. Using conventional mathematical extrapolation techniques, the Ford engineers then fitted a cost-speed curve to those three points, from which they derived their speed-frequency distribution using the narrower speed cells. Rohn Affidavit at 1a-2a. 84 Insurance Company Petitioners reiterate and expand the charge--made earlier by IIHS in its petition for reconsideration--that some of the assumptions underlying the agency's derivation of effectiveness values (Columns 7 and 8 of Table III) are questionable. Those values had been derived from so-called effectiveness curves constructed by the agency in 1979 on the basis of several critical assumptions: (1) that no bumper system provides complete protection at any speed; (2) that at their design speed, aluminum and steel bumpers are 60 percent effective; (3) that effectiveness falls to zero at replacement speed; and (4) that the effectiveness of bumpers is described by a curve of a particular shape. Department of Transportation, NHTSA, Calculations and Supporting Material for the Preliminary Analysis of the Bumper Standard at B-1 (Feb. 26, 1979), J.A. 1218 (NHTSA Calculations). By applying these assumptions to the available data for each bumper system, 19 three points were established on a speed/effectiveness graph (corresponding to replacement speed, design speed, and 0.0 mph), through which the appropriate curve was drawn. The effectiveness values of the system for given speeds were then read right off the curve; and the effectiveness values for given speed intervals were derived by estimating the area under each curve and calculating [243 U.S.App.D.C. 141] a mean value for each incremental speed range. Id. 85 The principal challenge to this analysis made by IIHS in its petition for reconsideration was that crash test data refuted the assumption that the 5.0 mph bumper loses all its effectiveness at its replacement speed (10 mph), showing that even Standard 215 5.0 mph bumpers resulted in substantial reductions in damage in barrier impacts of 10 and 15 mph. IIHS Petition for Reconsideration at 8, J.A. 20. NHTSA responded fully to this point, noting that the IIHS tests were conducted under laboratory conditions and therefore failed to reflect certain factors in real-world collisions, that IIHS's conclusions were based on a small number of vehicles, and that there was a great deal of year-to-year scatter in the data for individual vehicle types. 47 Fed.Reg. 56,649. More importantly, NHTSA concluded that if an adjustment were made for 5.0 mph bumpers, a comparable adjustment would have to be made for 2.5 mph bumpers. Id. 86 In this appeal for the first time, Insurance Company Petitioners seek to attack not merely the use of theoretical analysis (rather than crash test data) but the integrity of the analysis, describing the agency's initial assumptions of 60 percent effectiveness at design speed and zero effectiveness at replacement speed as guesswork, and asserting that the shape of the curves drawn through the three points (as described above), being unsupported by any equation in the rulemaking record, was arbitrary in the purest sense, Brief for Insurance Company Petitioners at 35. Such an ambush at this late stage cannot be allowed. The agency's analysis accompanying its final rule asserts that the assumptions were based on engineering judgment of the agency's experts, 47 Fed.Reg. 21,831. We have no reason to disbelieve that statement, and engineering judgment is assuredly the sort of expertise that NHTSA preeminently possesses. The only real issue, then, is whether the details of the agency's engineering analysis on these narrow points were so critical to the ability to participate in the rulemaking as to cause their absence from the record to invalidate it, see Portland Cement Ass'n v. Ruckelshaus, 486 F.2d 375, 392-94 (D.C.Cir.1973), cert. denied, 417 U.S. 921, 94 S.Ct. 2628, 41 L.Ed.2d 226 (1974). It is impossible to think so, since if they were there would have been strenuous objection much before this. Both the assumption of 60 percent effectiveness at design speed and the shape of the effectiveness curves have been a part of the agency's methodology since the January 1979 Preliminary Assessment and were carried forward (with minor modifications in light of IIHS comments) to the June 1979 Final Assessment, which the insurance industry found not only acceptable, but congenial. 87 The Insurance Company Petitioners' response to this reality is disingenuous. They assert that NHTSA is incorrect to state that it first heard of problems with its effectiveness curves in the IIHS petition for reconsideration, since IIHS responded to the ... 1979 [Preliminary] Assessment within a few months of its issuance and submitted alternative curves. Reply Brief for Insurance Company Petitioners at 18-19 n. 27. The statement is true, but neglects to note that the IIHS alternative curves displayed the same alleged defects which petitioners claim invalidate NHTSA's curves--i.e., were drawn through three points, one of which represented the assumption of 60 percent effectiveness at design speed. Indeed, IIHS adopted without change the agency's effectiveness curve for 5.0 mph bumpers. Comments of IIHS, Docket No. 73-19-N.25-097, App. C at 4 (Apr. 30, 1979), J.A. 2240. Insurance Company Petitioners boldly assert that no 'engineer' would attempt to draw a curve solely on the basis of only three points, id. at 18. But NHTSA did not purport to have designed the curve solely on the basis of three points. It drew the curve through the three points, but the shape of the curve was evidently determined by the location of those points and engineering theory. NHTSA Calculations at B-1, J.A. 1218. It is interesting that some engineer did the same thing in the Ford study of distribution [243 U.S.App.D.C. 142] of accidents by speed, discussed at pages 1359, supra. 88 The dissent's response is equally unsatisfactory, consisting essentially of the assertion that late presentation of the issue must be disregarded because validity of the curves is essential to validity of the agency's conclusion. Dissent at 1388-89. It is simply not the case, however, that all of the essential postulates for an agency rule must be contained in the record. Every judgment of any consequence is constructed upon an infinitude of other judgments, of greater or lesser certitude, in a progression of logical dependency terminating in a first principle the equivalent of 1 + 1 = 2. They cannot all possibly be included in the statement of basis and purpose for a rulemaking. We do not have authority to require that all elements underlying a rule be set forth in a fashion understandable to a layman, see Vermont Yankee Nuclear Power Corp. v. Natural Resources Defense Council, Inc., 435 U.S. 519, 557, 98 S.Ct. 1197, 1218, 55 L.Ed.2d 460 (1978). And it may as well be disclosed that in scientific fields we judges ourselves are laymen, ill equipped to determine where the line falls between requisite explanation of problematic analysis and useless replication of what for the cognoscenti amounts to a textbook on basic physics. Thus, our feel for the adequacy of an analysis in such cases is necessarily governed by the reaction that it elicits from knowledgeable commenters. From the point at which it reaches common ground we can reasonably assume that no further explication is required. We will hear on appeal assertions that needful elaborations fairly requested were not provided; but we must be implacably skeptical of belated recognition at the appellate stage that elements of scientific analysis unchallenged during a contested proceeding are incomprehensible without further explanation. To credit such post-appeal pleas of inadequate information is to threaten the integrity of all rulemaking in fields beyond our own limited scientific ken. The present challenge to the effectiveness curves presents the threat in a particularly flagrant form. NHTSA specifically asked the petitioners (and other rulemaking participants) in 1979: Do the existing analyses represent the most appropriate methods of approaching a study of bumper standards at different impact speeds and levels of damage resistance? If not, what method should be used? Comments of IIHS, Docket No. 73-19-N.25-097 at 5 (Apr. 30, 1979), J.A. 2200; Comments of State Farm, Docket No. 73-19-N.25-098 at 1 (Apr. 30, 1979), J.A. 2059. Though State Farm and IIHS, among others, responded with substantial comments--see Comments of IIHS, supra, Apps. A-B, J.A. 2212-35; Comments of State Farm, supra, 1-6, J.A. 2059-64--there was not even a suggestion that the effectiveness curves represented a fundamentally invalid methodology. The dissent provides a list of reasons why petitioners might not have raised these objections during the rulemaking proceeding, Dissent at 1387-88 all of which boil down to petitioners' probable reluctance to upset favorable determinations. Even if that were the only consideration, it would be of questionable wisdom to reward their tactical decision to leave the agency in the dark--and thus encourage benighted agency action in the future. But in any event, the dissent's speculations do not explain why the proponents of the 2.5 mph standard did not raise these arguments. In fact, some commenters on the June 1979 Final Assessment (which supported the 5.0 mph standard) did object that the effectiveness ratios for 5.0 mph systems are overstated to a considerable extent, Department of Transportation, NHTSA, Commentary on Critiques of the June 1979 Bumper Standard Assessment 35 (Dec. 1979), J.A. 951 (emphasis added), but that objection was based upon the lack of correspondence with repair cost data rather than theoretical inadequacy of the effectiveness curves. 89 We conclude that the curves were considered part of the common ground of expert analysis that required no further explanation. Petitioners' objections on this score put us in mind of the Supreme Court's injunction in Vermont Yankee that[243 U.S.App.D.C. 143] administrative proceedings should not be a game or a forum to engage in unjustified obstructionism by making cryptic and obscure reference to matters that ought to be considered and then, after failing to do more to bring the matter to the agency's attention, seeking to have that agency determination vacated on the ground that the agency failed to consider matters forcefully presented. 90 435 U.S. at 553-54, 98 S.Ct. at 1217. These instructions are even more appropriate when cryptic and obscure reference to theoretical inadequacy--or in fact, even less than that, mere assertion of lack of explanation of theoretical adequacy--is first made on appeal. 91 Insurance Company Petitioners also complain that, after correcting its initial assumption about the speed at which bumpers lose all effectiveness, in response to the IIHS comments (see note 19, supra ), NHTSA did not redraw the [effectiveness] curves, yet increased from 73 to 76 percent the presumed effectiveness of 2.5 mph bumpers in the 0.0-3.0 mph speed interval. Brief for Insurance Company Petitioners at 35-36. In fact, however, the change in the 0.0-3.0 mph speed cell was not the only change in effectiveness values made by the agency between February 1979 and the publication of the Final Assessment in June of that year. The figures for 2.5 mph and 7.5 mph metal bumpers were adjusted in higher speed cells, and substantial changes were made in the effectiveness values for soft-face bumpers. Compare NHTSA Calculations at B-2, J.A. 1220 with Final Assessment at 30, 32, J.A. 1011, 1013. The logical inference is that the agency did in fact rethink and redraw its curves following IIHS's suggestion, noted above, that the zero-effectiveness point be set at twice design speed.