Source: http://media.law.uark.edu/arklawreview/2014/03/20/et-tu-plaintiffs-an-empirical-analysis-of-dauberts-effect-on-plaintiffs-and-why-gatekeeping-standards-matter-a-lot/
Timestamp: 2017-09-25 18:37:25
Document Index: 85350589

Matched Legal Cases: ['§ 6', 'art, 96', 'art 2', 'art 117', 'art 116', 'art 115', 'art 104', 'art 103', '§ 6', '§ 2']

Et Tu, Plaintiffs? An Empirical Analysis of Daubert’s Effect on Plaintiffs, and Why Gatekeeping Standards Matter (a Lot) | Arkansas Law Review
66 Ark.L.Rev. 975 · download PDF
Andrew W. Jurs*
Scott DeVito**
[[975]]In the twenty years since the United States Supreme Court’s decision in Daubert v. Merrell Dow Pharmaceuticals, Inc.,1 researchers have tried to quantify the effect of the Court modifying the standard for admissibility of scientific evidence.2 Using different empirical methodologies, researchers reached contradictory results on whether Daubert created a stricter or more lenient standard. One analysis of reported opinions demonstrated Daubert had no effect, while another concluded the opposite.3 Survey results were similarly inconclusive.4 Just [[976]]this year, the authors of this article offered a new empirical analysis of Daubert’s effect.5 Our study postulated that the effect of the adoption of the Daubert scientific admissibility standard could be measured through aggregation and analysis of civil defendants’ decisions to remove their cases to federal court.6 Applying a fixed-effects statistical model to analyze a database of millions of actual cases, we found that civil defendants demonstrated by their actions that Daubert is the stricter gatekeeping standard.7 In effect, these defendants were more likely to try to enter federal court when they believed the standard was stricter after Daubert than before it—by a statistically significant margin.8 Yet, even after that study, significant questions remained, and we wondered if we could replicate the results using a different metric.9
In this article, we answer that question by applying the fixed-effects statistical model to a new and different dataset. Instead of measuring the removal decisions of civil defendants, we will measure the filing decisions of civil plaintiffs and compare their choices before and after Daubert’s adoption. By doing so, we can quantify whether civil plaintiffs demonstrate—through their actions in millions of cases—agreement with defendants that Daubert represents a stricter admissibility standard. After evaluating the data, we find that after federal courts adopted Daubert in 1993, civil plaintiffs increased the [[977]]rate at which they filed cases in a state court adhering to the older Frye standard.10 This data alone is enough to demonstrate that civil plaintiffs collectively act in ways demonstrating Daubert is a stricter standard.
In addition to civil plaintiffs’ initial reactions to Daubert, we can confirm our theory by demonstrating a counter-effect that occurs in the event a state changes to the same admissibility standard used in federal courts. When a state court adopts a Daubert-like standard that mirrors the federal standard, a plaintiff no longer receives a strategic advantage through choice of venue.11 When we evaluated the data after 1993, we found exactly this counter-effect: plaintiffs’ filing choices shift in those states that adopt Daubert after 1993, eliminating the post-Daubert change and returning to the pre-Daubert filing pattern.12 This counter-effect confirms that civil plaintiffs—through their collective behavior in millions of actual cases—act in ways showing Daubert is the stricter standard.
Our data analysis demonstrates that both civil plaintiffs and civil defendants act in ways affirming Daubert as the stricter standard. This finding has major policy implications. When the United States Supreme Court adopted the Daubert standard, the decision demonstrated a belief that the standard would be lower than before.13 In the 1997 decision General Electric Co. v. Joiner, the Court stated explicitly that the new Daubert standard would “allow district courts to admit a somewhat broader range of scientific testimony than would have been admissible under Frye.”14 Yet the data shows that the Court’s assumption of a [[978]]lower standard after Daubert is clearly invalid. Instead, the gatekeeping burden is higher. If so, we believe the Supreme Court must revisit the Daubert standard and select between two options: (1) acknowledge that the gatekeeping standard in federal court is strict and, therefore, affirm the reality that exists in courtrooms today; or (2) revisit the standard and command courts to apply the more lenient standard as the Supreme Court originally intended.15 The Supreme Court cannot, however, permit such a clear dichotomy between theory and practice to remain.
To examine these issues, we proceed as follows. We begin in Part II by reviewing prior empirical evaluations of Daubert’s effect, including our previous study measuring the behavior of civil defendants. This Part also evaluates the policy implications of a stricter Daubert and explains why data demonstrating a strict gatekeeping standard demands attention from the Court. In Part III, we present our empirical work by reviewing our data collection, our statistical evaluation of that data, and the results that demonstrate both an initial effect and a post-Daubert counter-effect on civil plaintiffs. Part IV concludes with brief comments on the implications of this study and revisits why a meaningful Daubert matters.
By measuring the behavior of civil litigants in actual cases, our analysis demonstrates that Daubert is a stricter gatekeeping standard. Consequently, we believe the theoretical underpinnings of Daubert have become unmoored from reality, meriting Supreme Court evaluation of the issue by accepting certiorarion a case involving expert gatekeeping.
II. Analyzing Daubert’S Effect: Prior Research and the Implications of a Finding of Stricter Admissibility
The standard resulting from Daubert has enormous implications for substantive law, access to justice, and the judicial system. In this Part, we will review researchers’ previous efforts to quantify Daubert’s substantive effect using [[979]]different empirical methodologies. In so doing, we will also review our own work in this area, explaining the methodology used to analyze Daubert’s effect on civil defendants. Lastly, this Part explores the implications of a stricter gatekeeping standard and explains why it leads to our conclusion that the Supreme Court must clarify the doctrine of expert admissibility.
A. Prior Research on Daubert’s Substantive Effect
Research into Daubert’s substantive effect has taken many forms, from surveys of judges applying Daubert to analysis of reported cases in computer databases like Westlaw. Unfortunately, the results of these studies have been largely inconsistent.
In 2001, Lloyd Dixon and Brian Gill evaluated Daubert’s substantive effect by reviewing reported civil decisions contained within a computerized database (Westlaw).16 To measure the change in gatekeeping since 1993, the researchers assessed how often judges scrutinized expert evidence and how often they excluded it before and after Daubert.17 When they did so, they determined the data “suggest[s] that the standards for reliability tightened in the years after the Daubert decision.”18 Yet the researchers were careful to note that the effect changed over time, with a larger impact in the period immediately after Daubert to a lesser one a few years afterward.19 Ultimately, they concluded that, “following Daubert, judges scrutinized reliability more carefully and applied stricter standards in deciding whether to admit expert evidence.”20
However, in 2002, Jennifer Groscup and her research group conducted a computer-database analysis that found the opposite: Daubert had not changed the rate of admissibility for expert evidence.21 Just as Dixon and Gill had, Groscup’s team [[980]]evaluated appellate decisions reported in a computerized database (Westlaw).22 They then measured the frequency with which certain Daubert-related terms appeared in criminal-case decisions and determined the rate of expert-admissibility at both trial and on appeal.23 After measuring and coding the terms within the cases, Groscup and her group concluded: “[W]e were unable to detect any major changes in the admission of expert testimony as a result of the Daubert opinion.”24 Yet even with that conclusion, they still believed Daubert affected the scrutiny judges applied—even without changing the admissibility rates over time.25 Clearly, between Groscup and Dixon and Gill, the methodology of computerized-database analysis leaves the central question of Daubert’s substantive effect unanswered.
Instead of computer-database searches, other researchers have tried to measure Daubert’s effect by surveying the gatekeepers—i.e., the judges who make the decisions on admissibility of expert evidence. Mirroring Dixon and Gill’s finding, Carol Krafka and her group found—based on survey data from federal judges in the 1990s—that Daubert made judges more likely to scrutinize and exclude expert evidence.26 Their survey received responses from over 300 federal judges about the procedures and substantive factors they used in expert gatekeeping.27 A large majority of judges reported that their procedures for assessing expert evidence had not changed significantly since Daubert, but on its substantive effect, one-third of judges stated they were less likely to admit expert evidence after Daubert.28
In contrast to Krafka’s findings, other survey work mirrors Groscup’s findings and concludes Daubert has no clear substantive effect on the standard for admissibility. A 2001 [[981]]study of state-court judges by Sophia Gatowski and her coauthors reaches that conclusion.29 In their work, the researchers surveyed 400 state-court judges about evidentiary gatekeeping.30 When directly asked whether Daubert represents a higher or lower standard for admissibility, the judges did not provide a clear consensus: 32% stated Daubert raised the standard of admissibility; 23% thought it lowered the standard; and 36% stated Daubert neither raised nor lowered the standard.31 This data supports the contention that even if judges take an active procedural role in gatekeeping—as 62% of the judges stated they did32—they do not consistently hold expert evidence to a higher substantive standard. Just as with computerized-database research, survey research has also not answered the central question of the substantive effect of Daubert.
Considering these inconclusive results, we decided that if we wanted to measure Daubert’s substantive effect, we could not rely on reported case decisions or on surveys of judges. Instead, we decided to measure how litigants act in real-world cases by measuring the aggregate effect of Daubert on case-management decisions.33 Our first study measured the decisions of civil defendants to remove their cases to federal court.34 The central premise underlying removal-rate analysis is that courts consider evidentiary rules—such as Daubert’s admissibility standard—procedural under the Erie doctrine.35 Thus, the venue switch will change the scientific admissibility standard when the state court in question continues to apply the Frye standard,36 which the Supreme Court rejected in Daubert.37 Of course, [[982]]defendants will choose to remove their cases only when they perceive this strategy as beneficial.
Once we decided on removal rate as the metric for analyzing Daubert’s effect, we needed a database of actual cases with which to measure any changes in removal rates after the Court decided Daubert in 1993. Using publically available data on state- and federal-court filings, we amassed amassed a collection of nearly four million individual cases during the period from 1990 to 2000.38 We then applied a fixed-effects statistical model to that data,39 leading us to two major findings. First, we found that after the federal adoption of Daubert in 1993, civil defendants were more likely to remove a case to federal court than they were prior to 1993.40 Since defendants will only remove a case when it benefits them, we concluded that this finding demonstrates civil defendants—in the aggregate, based on millions of cases—believe Daubert is the stricter standard.41 That finding is significant, but we wanted to go further.
The next stage of our analysis examined the effect of a state court adopting the Daubert standard after 1993.42 Since states are free to choose which standard to follow, many states since 1993 have adopted the Supreme Court’s Daubert standard for gatekeeping.43 But when a state court adopts Daubert, civil defendants lose the benefit they previously gained from venue-shifting from state to federal court.44 When we analyzed the changes in removal rates in the years after 1993, we found that the removal rate will decrease once a state adopts Daubert.45 This finding is exactly what one would expect to see if civil defendants believe Daubert is the stricter standard. Based on our analysis of civil defendants, we then believed that statistical modeling is the best way to analyze Daubert’s substantive effect. Using aggregate case data in millions of actual cases, our [[983]]analysis provided strong evidence that Daubert changed the standard for experts and that it is a stricter standard than the alternative Frye standard.
Once we found that civil defendants believe Daubert is a stricter standard, the next step became clear: measuring the effect of Daubert on civil plaintiffs by a similar assessment of their collective behavior. When we analyze Daubert’s effect on civil plaintiffs in Part III, we confirm that it heightened the substantive standard for gatekeeping after 1993.46
B. Implications of a Stricter Daubert Admissibility Standard
Before proceeding to this article’s mathematical analysis of how civil plaintiffs’ filing choices demonstrate Daubert’s heightened gatekeeping standard, we need to review why a stricter standard is a finding of enormous significance that merits Supreme Court review. In essence, the empirical findings of a strict substantive standard lead to one inalterable conclusion—a court’s choice of an evidentiary gatekeeping standard is a choice between strict and lenient gatekeeping alternatives; thus, it is a choice that makes a difference in the administration of justice across the United States.
One reason the choice matters is because substantive changes in gatekeeping can affect an enormous number of cases in the judicial system. Data on the use of expert witnesses throughout the court system vary, but studies consistently show that a clear majority of cases involve some type of expert.47 In one study, 86% of cases going to trial involved expert testimony!48 If these numbers remain true today, even a small [[984]]change in expert admissibility standards has an enormous effect on the judicial system. By the numbers, changes to gatekeeping of experts could affect tens of thousands of the more than 294,000 civil cases filed in federal court annually49—particularly the 3200 that go to trial each year.50 Of course, these number totals rise significantly if they also include the nearly eight million civil cases filed in state court each year.51 Few issues in the justice system have such a profound, system-wide impact, meriting close scrutiny of any change in the law.
A second reason why a court’s choice of an expert admissibility standard matters, beyond the sheer volume of cases, is the effect of even minor changes in the gatekeeping standard on substantive tort law. Generally, in our civil-justice system, a plaintiff filing a case has the burden of establishing the necessary elements to succeed on a claim.52 Since so many cases involve expert testimony, even a small shift in expert admissibility standards will necessarily raise the burden on those claimants. Of course, many may argue that increasing burdens is an appropriate public policy and would benefit the justice system.53 Our only contention is that such changes should be debated and adopted directly—not through backdoor maneuvers—to ensure the legitimacy of the results.
The third policy implication of an empirical finding that Daubert represents a stricter gatekeeping standard is that the actual effect of the standard has become unhinged from the rationale underlying its adoption. In Daubert, Justice Blackmun and the majority clearly believed that the new post-Frye [[985]]reliability standard they adopted would be a more lenient standard.54 One reason for the Court’s adoption of the new Daubert standard, after all, was that the “austere” Frye standard of “‘general acceptance’” was incompatible with the liberal approach of admissibility throughout the Federal Rules of Evidence.55 Blackmun even directly noted that, when faced with weak expert evidence, courts should admit the evidence and allow the adversarial system to work (i.e., by allowing the opponent to point out its flaws).56 Four years later, the Court, in General Electric Co. v. Joiner, specifically stated what it implied in Daubert—that “the Federal Rules of Evidence allow district courts to admit a somewhat broader range of scientific testimony than would have been admissible under Frye.”57
Yet when we look at the data, we see that the real-world effect of Daubert is exactly the opposite. Civil defendants, seeing that Daubert will benefit them through stricter scrutiny of plaintiffs’ experts, are more likely to want to switch to a Daubert venue when plaintiffs file in a Frye state court.58 As we shall demonstrate in Part III, civil plaintiffs act accordingly, shifting the filing of their cases to more attractive venues after deeming Daubert the stricter standard.59 If so, the data tells a story we cannot afford to ignore: Daubert has the effect of limiting expert testimony, and litigants know it.
We believe the Supreme Court cannot permit the current situation to continue and must choose from several alternatives. The Court could choose to reinvigorate the original intent of Daubert by commanding courts to apply Blackmun’s liberal standard for admissibility.60 Alternatively, the Court could affirm the situation that exists in courtrooms today and decide gatekeeping should apply the strict substantive standard that actually resulted from Daubert. Finally, the Court could always consider striking a fresh approach, offering new instructions on the substantive rules of gatekeeping. In choosing between its [[986]]options, however, the Court must reconcile the reality in courtrooms today with the intent of Blackmun’s lenient admissibility standard. We believe the data compels the Supreme Court to accept certiorari on a case, revisit Daubert gatekeeping, and clarify the substantive standard for expert admissibility.
Research into factors affecting the Supreme Court’s certiorari decisions supports our assertion. In 1988, Gregory Caldeira and John Wright discussed four factors affecting the Court’s decision to accept certiorari: (1) a conflict between circuits; (2) the United States as petitioner; (3) ideological disagreement; or (4) unusual practical importance.61 Caldeira and Wright also theorized that one could gauge the practical import of an issue by the number of amicus filings accompanying the certiorari petition, which can signal the potential impact of a decision on the issue.62 When they reviewed the data, they found that even one amicus filing raises the likelihood of judicial review significantly and that additional briefs filed after the first improve the chance for review even more.63 In fact, the single factor of two or three amicus briefs increased the probability of the Court accepting certiorari by at least 34%.64 Considering the import of gatekeeping as measured by amicus filings, Caldeira and Wright’s study has direct importance to the Daubert issue in this article.
The most recent petition for certiorari that raised questions about the substantive standard for expert gatekeeping was in the [[987]]2011 case United States Steel Corp. v. Milward.65 In Milward, the First Circuit endorsed a Blackmun-like approach to gatekeeping,66 finding: “So long as an expert’s scientific testimony rests upon ‘good grounds’ . . . it should be tested by the adversarial process, rather than excluded for fear that jurors will not be able to handle the scientific complexities.”67 After this decision, the defendants petitioned the Supreme Court for certiorari, asking that the Court exclude the expert’s testimony based on a more restrictive view on gatekeeping.68 When they did so, no fewer than three organizations filed amicus briefs in support of the defendants’ petition.69 Although the Court rejected certiorari,70 the filing of those briefs signaled to the Court the importance of the issue, increasing the likelihood of judicial review under Caldeira and Wright’s analysis. If amicus filings signal to the Court the importance of an issue, and since the last Daubert-decided case had many amicus filings, then our conclusion that the Supreme Court should accept certiorari seems less about speculation than about timing.
We believe that the Court should, in a Milward-type case, accept the invitation to clarify the substantive standard for expert-witness admissibility. Such a result is the clear implication of our empirical work finding that, since 1993, judicial gatekeeping has applied a stricter standard than the Court intended in Daubert.
III. Empirical Analysis of Plaintiff Filing Patterns
[[988]]Having reviewed the enormous importance of the issue of substantive changes to expert admissibility and researchers’ prior efforts to quantify the change in standards since Daubert, we now turn to our empirical analysis of plaintiff filing patterns. Our analysis tests whether attorneys representing plaintiffs in civil cases act in ways consistent with Daubert being a stricter standard than the alternatives.71 The key to this analysis is quantifying attorneys’ opinions by aggregating and assessing their collective judgment on how Daubert works. Such an analysis produces a clear result: Just as with civil defendants,72 civil plaintiffs believe Daubert is a stricter standard for expert gatekeeping.
A. Thought Experiment: How Would Plaintiffs Act If Daubert Is a Stricter Standard?
In order to measure whether Daubert is a stricter standard, we must first understand how the data might look with Daubert as a stricter standard. Doing so requires remembering that attorneys file where they think they can win.73 When attorneys rely on expert testimony in their cases, they may have the option to file in federal or state court. Because federal and state courts have different gatekeeping standards, all else being equal, attorneys will likely file in the jurisdiction with the weakest gatekeeping standard—i.e., the one with the lowest probability of excluding the evidence.74 In short, when a gatekeeping gradient exists—i.e., an advantage in gatekeeping that is accessible through venue choice—plaintiffs’ attorneys should file in the venue with the weaker gatekeeping standard in order to increase their chances of success.75
By observing where attorneys file their cases, we can use the creation and destruction of the gatekeeping gradient to determine whether attorneys act as if Daubert is the stricter standard. For example, assume that before 1993 both the federal and state courts in State A applied the pre-Daubert, federal standard of “general acceptance” under Frye.76 Assume further that in 1993—when Daubert changed the federal gatekeeping standard77—State A’s state courts continued to use the Frye standard. Thus, beginning in 1993, State A’s federal and state courts applied different gatekeeping standards for expert evidence, creating a gatekeeping gradient.
Prior to 1993, filing in federal court, rather than state court, did not yield a Daubert–Frye evidentiary standard type of gatekeeping advantage. Therefore, before 1993 the probability of filing in federal court should appear fairly constant.78 But [[990]]when the gatekeeping systems are different—as they would be in our post-Daubert hypothetical—plaintiffs’ attorneys could choose to file in either a state court applying Frye or a federal court applying Daubert. If attorneys believe that Daubert applies a stricter gatekeeping standard, the data should demonstrate a shift in where attorneys file their cases as plaintiffs’ attorneys choose to remain in state court because of its more lenient standard. This effect should, relative to the pre-1993 period, produce a decrease in the probability of filing in federal court. Graphically, the result would look similar to Figure 1—the “Flight to Frye” Theory.
Figure 1. The “Flight to Frye” Theory: Effect on Filings in State A from 1990 to 1997
On the other hand, if plaintiffs’ attorneys believe that application of Daubert and Frye have nearly identical effects on [[991]]the exclusion of evidence, the probability of filing in federal court should remain stable—a generally flat pattern. Or, if those attorneys view Frye as the more restrictive standard, the probability of filing in federal court should increase—an upward-shifting pattern after 1993. In either case, one would not expect to see a downward-shifting pattern like Figure 1.
The destruction of the gatekeeping gradient provides another opportunity to determine whether attorneys believe Daubert is a stricter standard. Assume that after the federal courts adopted Daubert in 1993, State B retained the Frye gatekeeping standard until finally adopting the Daubert gatekeeping standard in 1997. In this scenario—although State B would have experienced a decrease in filings in federal court after the federal courts adopted Daubert in 1993—plaintiffs’ attorneys, by 1997, could no longer take advantage of the gatekeeping gradient through venue choice because it would not exist. Having lost this advantage, the probability of plaintiffs filing in federal court should increase to a level similar to the previous period—returning many of the losses occurring after 1993 to federal court. One may refer to this effect as the “Return to Federal Court” Theory, and graphically, it would look similar to Figure 2.
Figure 2. The “Return to Federal Court” Theory: Effect on Filings in State B from 1994 to 2001
[[992]]Once again, if plaintiffs’ attorneys believe that application of Daubert and Frye have nearly identical effects on the exclusion of evidence, the probability of filing in federal court should remain stable—a generally flat pattern. However, if these attorneys believe Frye is the more restrictive standard, one would expect the probability of filing in federal court to decrease—a downward-shifting pattern like Figure 1—after the state court adopts Daubert. In either case, the upward-shifting pattern in Figure 2 should not occur.
As the following section will demonstrate, the changes depicted in Figure 1 and Figure 2 actually occurred as litigants in real cases reacted to the shifting gatekeeping standards in federal and state court. This effect confirms that plaintiffs’ attorneys believe—as evinced by their actions—the Daubert standard is stricter than the Frye standard.
B. Analysis of Case Data Shows That Adoption of Daubert Affects Plaintiffs’ Venue Choices
To test the validity of the “Flight to Frye”and “Return to Federal Court”theories, we first created a database of cases filed in the relevant period. Next, we statistically analyzed that database to measure plaintiffs’ reactions as different venues adopted Daubert. The following subsection explains the creation of this database and the method for analysis. The subsection concludes by discussing the results of the analysis.
We constructed the database using publicly available, court-statistic databases created by the National Center for State Courts and the Federal Judicial Center, which are accessible at the Inter-university Consortium for Political and Social Research (ICPSR).79 While these court-statistic databases [[993]]contain many fields, our database contains only three fields: (1) the year; (2) the state; and (3) where the plaintiff filed the case (federal or state court).80 This data was further limited to a group of thirteen states and to a period of thirteen years for which complete and reliable data exists.81 Finally, we limited the data compiled for each state and each year to tort suits filed in those thirteen states for the thirteen-year period.
For example, in Kansas in 1997, plaintiffs filed 6194 tort suits in state court82 and 227 tort suits in federal district court.83 [[994]]Thus, the database contains a total of 6421 entries for Kansas in 1997—227 federal filings84 and 6194 state filings.85 Applying this procedure to the thirteen states used in the study for the years 1990 through 2002 produces a database with nearly 3.5 million entries.86
We then created two datasets from this database. One dataset corresponds to the “Flight to Frye” period and the other corresponds to the “Return to Federal Court” period.87 To create the “Flight to Frye” dataset, we began with thirteen states88 and [[995]]eleven years.89 Because we assumed transition years would be confusing and could potentially distort the analysis,90 we eliminated all data relating to 1993. In addition, because some of the states adopted Daubert between 1993 and 2000,91 this study also eliminated data relating to those states from the year they adopted Daubert.92 Finally, to limit the effect of unaccounted for variables that might interfere with the analysis,93 we limited the data to the three years before and after 1993.94
In creating the “Return to Federal Court” dataset, this study began with thirteen states.95 Because this study analyzes the effects of states adopting Daubert after federal courts adopted Daubert, we limited the years to the period after federal adoption of Daubert.96 To create an appropriate set of controls, we kept the data from all nine years for the pure-Frye states during this period.97 To limit the effect of unknown covariates, [[996]]for any state that adopted Daubert during this period, we limited the data to the period three years before and three years after the state adopted Daubert.98
2. Analysis of Data Using a Fixed-Effects Approach
We designed this study to determine if either state or federal-court adoption of Daubert changed the probability that attorneys will file their cases in a particular court. This required further dividing the datasets for both the “Flight to Frye” and “Return to Federal Court” datasets into two comparison classes.
For the “Flight to Frye” dataset, the division occurred in 1993 when all federal courts adopted Daubert.99 We could then measure the probability of filing in federal court for the pre-1993 group and compare it with the probability of filing in federal court for the post-1993 group to determine if there was any statistically significant100 correlation between the federal-court adoption of Daubert and any change in the probability of filing in federal court.101
[[997]]For the “Return to Federal Court” dataset, because every state’s federal courts used Daubert, we divided the dataset into one group where the state court adopted Daubert and another group where the state court did not adopt Daubert.102 This division allowed us to measure the probability of filing in federal court in each group and to conclude whether a statistically significant correlation exists between state-court adoption of Daubert and any change in the probability of filing in federal court. If we found the right kind of change in the probability of filing in federal court, there would be a strong argument that plaintiffs’ attorneys perceived Daubert as the stricter standard.103
As a further effort to avoid the consequences of unknown covariates, this study employs a logistic,104 fixed-effects analysis to measure any correlation between choice-of-filing venue and adoption of Daubert. The advantage of a fixed-effects analysis is that it captures at least some of those potential unknowns. A fixed-effects analysis105 is performed on a regression formula containing a set of dummy variables106 that are designed to [[998]]capture individual-specific unknowns.107 Our study structures the regression formula108 to contain dummy variables to account for state- and year-specific unknowns.109
The fixed-effects analysis of the “Flight to Frye” and “Return to Federal Court” groups produced statistically significant effects that support this study’s hypothesis—that Daubert is the stricter standard. In the case of the “Flight to Frye” group, the probability of filing in federal court decreased by nearly 21% when the federal courts began to use Daubert:
Table 1. Effect of Daubert on the Probability of Filing in Federal Court110
Daubert Adopted by Federal Courts
Probability of Filing in Federal Court
No 0.014113
Yes 0.011683
[[999]]To ensure that a shift in population over the ten-year period did not cause these effects, we normalized the data to eliminate population growth as a factor.111 Using this normalized data, the fixed-effects analysis produced a similar 20% decrease in the probability of filing in federal courts when the federal courts began to use Daubert:
Table 2. Effect of Daubert on the Probability of Filing in Federal Court—Data Normalized for Population Growth112
No 0.014128
Yes 0.011750
These results match precisely what one would expect if attorneys believe that Daubert is a stricter standard than Frye and depict a situation just like Figure 1.
In the case of the “Return to Federal Court” group, the probability of filing in federal court increased by nearly 22% when the state courts adopted Daubert a few years after the federal courts began to use Daubert:
Table 3. Effect of State-Court Adoption of Daubert on the Probability of Filing in Federal Court113
Daubert Adopted by State Court
No 0.011551
Yes 0.014065
After normalizing the data to account for population growth, we found a nearly 17% increase in the probability of [[1000]]filing in federal court given that the state court also applied Daubert114:
Table 4. Effect of State-Court Adoption of Daubert on the Probability of Filing in Federal Court—Data Normalized for Population Growth 115
No 0.011867
Yes 0.013835
The rise in probability matches the results one would expect if attorneys believe that Daubert is a stricter standard, and the results depict a situation just like Figure 2. Moreover, the rise in probability corresponds closely to the decrease in probability caused during the “Flight to Frye” period, further supporting the view that state adoption of Daubert in the “Return to Federal Court” period eliminated the advantage that drove attorneys to file in federal court during the “Flight to Frye” period. These results strongly support the view that attorneys believe Daubert is a stricter standard than Frye and that their filing choices changed pursuant to that belief.
Since the United States Supreme Court decided Daubert in 1993, researchers have worked on quantifying its effect on the substantive standard for gatekeeping. Earlier studies used different methodologies and collectively remained inconclusive on the ultimate issue: whether Daubert raised or lowered the standard.
We believe litigants’ choices in real-world cases— measured in the aggregate in millions of actual lawsuits—provide an answer to that question. In an earlier work, we measured the effect of Daubert on civil defendants, [[1001]]demonstrating that these defendants act in ways evincing a belief that Daubert produced a stricter gatekeeping standard for expert evidence than Frye did. In this study, we applied a fixed-effects statistical analysis to the choices of civil plaintiffs, again measuring their choices in real cases in order to quantify the substantive effect of Daubert. When we examined the data, we found that civil plaintiffs also reacted to Daubert in a manner consistent with a belief that it represents a stricter gatekeeping standard than Frye. Therefore, our analysis demonstrates that both sides perceive Daubert as the stricter standard.
If the substantive standard under Daubert is higher, there are enormous public-policy implications to consider. The sheer number of cases involved is enormous, and a stricter standard involves a subtle shifting of substantive tort law. But perhaps most importantly, the finding that Daubert is the stricter standard proves that the policy underlying the Daubert decision has become disconnected from its true effect. Since the Court intended Daubert to be a more lenient standard, and because the actions of attorneys in millions of cases reveal that it is not, the Court should re-address the expert admissibility standard. Either the Court will affirm the reality of the stricter standard or it must reassert that a judge must perform his gatekeeping role consistent with the “liberal thrust” of the rules. The current situation—with the policy underlying the standard and the standard’s effect being diametrically opposed—cannot continue.
By measuring the actual behavior of civil plaintiffs, this article answers the question whether Daubert is the stricter standard for expert admissibility and finds that it is. If so, we believe the theoretical underpinnings of Daubert have become unmoored from reality, meriting Supreme Court evaluation of the issue by accepting certiorarion a case revisiting the substantive standard for gatekeeping.
[[1002]]where the subscript i identifies the state (so i = 1 when the state is Alaska, 2 if the state is Arizona, etc.);
the subscript t identifies the year (so t = 1 for 1990, 2 for 1992, etc.);
the subscript n identifies this as the nth tort filed in state i and year t;
is a 0 if the nth tort in year t and state i was filed in state court and 1 otherwise;
is a set of binary variables such that is set to 1 when i is 1 (the state is Alaska) and to 0 otherwise, is set to 1 when i is 2 (the state is Arizona) and to 0 otherwise, etc.;
is a set of binary variables such that is set to 1 when t = 1 (the year is 1990) and to 0 otherwise, is set to 1 when t = 2 (the year is 1991) and to 0 otherwise, etc.;
is a binary variable that is set to 1 if state i at time t has adopted Daubert and zero otherwise, depending on the population to be studied, may correspond to federal adoption of Daubert or state adoption of Daubert;
is the y intercept;
is the regression coefficient for state i;
is the regression coefficient for year i;
is the regression coefficient for the Daubert variable; and
is the standard error term.
* Associate Professor of Law, Drake University Law School; J.D. 2000, University of California, Berkeley, School of Law; B.A. 1997, Stanford University.
**Associate Professor of Law, Florida Coastal School of Law; Ph.D. 1996, University of Rochester, Department of Philosophy; J.D. 2003, University of Connecticut; B.A. 1991, Queens College.
[1]. 509 U.S. 579 (1993).
[2]. See, e.g., David H. Kaye, David E. Bernstein & Jennifer L. Mnookin, The New Wigmore: A Treatise on Evidence § 6.4.1, at 223-25 (2004).
[3]. Compare Lloyd Dixon & Brian Gill, RAND Inst. for Civil Justice, Changes in the Standards for Admitting Expert Evidence in Federal Civil Cases Since the Daubert Decision xv (2001) (“Our analysis of district court opinions suggests that after Daubert, judges scrutinized reliability more carefully and applied stricter standards in deciding whether to admit expert evidence.”), with Jennifer Groscup et al., The Effects of Daubert on the Admissibility of Expert Testimony in State and Federal Criminal Cases, 8 Psychol. Pub. Pol’y & L. 339, 363 (2002) (finding that analysis of appellate decisions demonstrated “no change in the overall rate of admission for all types of expert evidence”).
[4]. Compare Sophia I. Gatowski et al., Asking the Gatekeepers: A National Survey of Judges on Judging Expert Evidence in a Post-Daubert World, 25 Law & Hum. Behav. 433, 433-34 (2001) (reporting that judges do not agree on the substantive effect of Daubert), with Carol Krafka et al., Judge and Attorney Experiences, Practices, and Concerns Regarding Expert Testimony in Federal Civil Trials, 8 Psychol. Pub. Pol’y & L. 309, 322 (2002) (“Compared with 1991, judges in 1998 reported that they were more likely to scrutinize expert testimony before trial and were less likely to admit it.”).
[5]. Andrew W. Jurs & Scott DeVito, The Stricter Standard: An Empirical Assessment of Daubert’s Effect on Civil Defendants,62 Cath. U. L. Rev. 675 (2013).
[6]. The central premise allowing the analysis of aggregate removal rates is that courts generally deem evidentiary rules procedural rather than substantive; thus, a state’s evidentiary rules on expert admissibility will not transfer to federal court upon removal. Id. at 679; see also Abbe R. Gluck, Intersystemic Statutory Interpretation: Methodology as “Law” and the Erie Doctrine, 120 Yale L.J. 1898, 1979 n.279 (2011) (noting that the Federal Rules of Evidence are procedural). Instead, when a defendant removes a case to federal court, the Federal Rules of Evidence will apply to the action even though state substantive tort law remains. Edward K. Cheng & Albert H. Yoon, Does Frye or Daubert Matter? A Study of Scientific Admissibility Standards, 91 Va. L. Rev. 471, 483 (2005) (“[P]arties litigating tort claims in federal court are governed by the same substantive tort law as those in state court.” (citing Erie R. Co. v. Tompkins, 304 U.S. 64, 78 (1938))); Jurs & DeVito, supra note 5, at 679 n.14.
[7]. See Jurs & DeVito, supra note 5, passim.
[8]. Id. at 693. As demonstrated with the plaintiff data in Part III.B, the data on defendants also showed that this effect moderates after state adoption of Daubert. Id.; see infra Part III.B.
[9]. See Jurs & DeVito, supra note 5, at 730.
[10]. See infra Part III.B (containing the methodology and analysis leading to this conclusion); see also Jurs & DeVito, supra note 5, at 682 (“Frye required that—before admission—a method be generally accepted in the relevant field.” (citing Frye v. United States, 293 F. 1013, 1014, (D.C. Cir. 1923), overruled by Daubert v. Merrell Dow Pharms., Inc., 509 U.S. 579 (1993))).
[11]. This strategic advantage no longer exists because, under Daubert, the federal venue and state venue will now have the same scientific admissibility standard. See Jurs & DeVito, supra note 5, at 679 n.14.
[12]. See infra Part III.B (containing the methodology and analysis leading to this conclusion).
[13]. The Daubert Court noted that the previousgeneral-acceptance admissibility standard from Frye was “austere” and “absent from and incompatible with the Federal Rules of Evidence” and their liberal admissibility standard. Daubert, 509 U.S. at 588-89 (citing Frye, 293 F. at 1014).
[14]. 522 U.S. 136, 142 (1997) (emphasis added).
[15]. See infra Part II.B.
[16]. Dixon & Gill, supra note 3, at 15-24 (describing the researchers’ data collection and study methodology).
[17]. Id. at 24-32.
[18]. Id. at 29.
[19]. Id. at 27 tbl.4.1, 31.
[20]. Id. at 61.
[21]. Groscup et al., supra note 3, at 363.
[22]. Id. at 342-44 (describing the team’s methodology of case selection and coding). Groscup’s database contains criminal cases while Dixon & Gill’s contains only civil cases; thus, the two studies address entirely different datasets. Id. at 343; Dixon & Gill, supra note 3, at xiii.
[23]. Groscup et al., supra note 3, at 343-45.
[24]. Id. at 370.
[25]. Id. at 363.
[26]. Krafka et al., supra note 4, at 322, 328-29.
[27]. Id. at 312.
[28]. Id. at 329.
[29]. Gatowski et al., supra note 4, at 443 (noting that judges conflicted in their beliefs on Daubert’s substantive effect).
[30]. Id. at 441.
[31]. Id. at 443.
[33]. See Jurs & DeVito, supra note 5, at 730 (discussing the benefit of aggregate case analysis).
[34]. Id. at 678-79 (providing a general review of the study’s methodology and results).
[35]. See supra note 6.
[36]. See Jurs & DeVito, supra note 5, at 679.
[37]. Daubert v. Merrell Dow Pharms., Inc., 509 U.S. 579, 597-98 (1993).
[38]. Jurs & DeVito, supra note 5, at 679.
[39]. Id. at 693.
[40]. Id. at 698-99 figs.3 & 4.
[41]. Id. at 701-02.
[43]. Jurs & DeVito, supra note 5, at 701 n.140.
[44]. Id. at 680.
[45]. Id. at 701 fig.5.
[47]. See Anthony Champagne et al., An Empirical Examination of the Use of Expert Witnesses in American Courts, 31 Jurimetrics J. 375, 380 (1991) (finding an expert in fifty-seven of the ninety civil cases tried in Dallas County District Court in 1988); Samuel R. Gross, Expert Evidence, 1991 Wis. L. Rev. 1113, 1119 (1991) (noting that experts testified in 86% of “529 civil trials that led to jury verdicts in California State Superior Courts in 1985 and 1986”); Daniel Shuman et al., An Empirical Examination of the Use of Expert Witnesses in the Courts — Part II: A Three City Study, 34 Jurimetrics J. 193, 197 (1994) (reporting that about 71% of 183 civil and criminal trials used experts).
[48]. Gross, supra note 47.
[49]. Administrative Office of the U.S. Courts, Federal Judicial Caseload Statistics tbl.C-2 (2011), available at http://www.uscourts.gov/Statistics/FederalJudicialCaseloadStatistics/FederalJudicialCaseloadStatistics2011.aspx.
[50]. Id. at tbl.C-4. These numbers, of course, would increase if they accounted for the effect on the criminal docket in federal court, adding nearly 80,000 cases filed and 2778 trials in 2011. See id. at tbls.D-1 & D-4.
[51]. Nat’l Ctr. for State Courts, State Court Caseload Statistics: Reported National Civil and Criminal Caseloads for State Trial Courts (2010), http://www.courtstatistics.org/other-pages/statecourtcaseloadstatistics.aspx.
[52]. Arthur v. Unkart, 96 U.S. 118, 122 (1877).
[53]. See, e.g.,Gary Edmond & David Mercer, Trashing “Junk Science,” 1998 Stan. Tech. L. Rev. 3, ¶¶ 4–8 (reviewing history of the “junk science” debate and discussing why it led to calls for more active scrutiny of proffered expert testimony).
[54]. See Daubert v. Merrell Dow Pharms., Inc., 509 U.S. 579, 588 (1993).
[55]. Id. at 588-89 (citing Frye v. United States, 293 F. 1013, 1014 (D.C. Cir. 1923)).
[56]. Id. at 596.
[57]. 522 U.S. 136, 142 (1997) (emphasis added).
[58]. Jurs & DeVito, supra note 5, at 699 figs.3 & 4, 701.
[59]. See infra Part III.B.
[60]. See Daubert, 509 U.S. at 588-89.
[61]. See Gregory A. Caldeira & John R. Wright, Organized Interests and Agenda Setting in the U.S. Supreme Court, 82 Am. Pol. Sci. Rev. 1109, 1114 (1988).
[62]. Id. at 1112.
[63]. Id. at 1119.
[64]. Id. at 1121 tbl.2. The 34% increase was the lowest increase for the increase from zero factors to the sole factor of two or three amicus briefs in support of certiorari. Id. The increase in likelihood of certiorari based on amicusfilings is even greater when any other factor is present: A 55% increase occurs with a conflict between courts of appeals, and a 59% increase occurs with the United States as the petitioner for certiorari. Calderia & Wright, supra note 61. There were no cases with all three factors present. Id. The factors mentioned by Caldeira and Wright were more recently examined by Kevin Smith in his 2001 law review article. Kevin H. Smith, Certiorari and the Supreme Court Agenda: An Empirical Analysis, 54 Okla. L. Rev. 727, 742-43 (2001). Smith found that filing an amicus brief increases the likelihood of the Supreme Court granting certiorari, affirming Caldeira and Wright’s finding. Id. at 754 n.116.
[65]. Milward v. Acuity Specialty Prods. Grp., 639 F.3d 11, 13-14 (1st Cir. 2011), cert. granted, 132 S. Ct. 1002 (2012).
[66]. See Milward, 639 F.3d at 15.
[67]. Id. at 15 (emphasis added) (citation omitted) (internal quotation marks omitted) (quoting Daubert v. Merrell Dow Pharms., Inc., 509 U.S. 579, 596 (1993)).
[68]. Petition for a Writ of Certiorari at 16-17, U.S. Steel Corp. v. Milward, 132 S. Ct. 1002 (2012) (No. 11-316), 2011 WL 4073080 at *16-17.
[69]. Brief for the Chamber of Commerce of the United States, American Chemistry Council et al. as Amici Curiae in Support of Petitioners, U.S. Steel Corp. v. Milward, 132 S. Ct. 1002 (2012) (No. 11-316), 2011 WL 4912891; Brief for DRI—the Voice of the Defense Bar as Amicus Curiae Supporting Petitioners, U.S. Steel Corp. v. Milward, 132 S. Ct. 1002 (2012) (No. 11-316), 2011 WL 4912893; Brief of Amicus Curiae, Product Liability Advisory Council, Inc., in Support of Petitioner’s Petition for Writ of Certiorari, U.S. Steel Corp. v. Milward, 132 S. Ct. 1002 (2012) (No. 11-316), 2011 WL 4912892.
[70]. Milward, 132 S. Ct. at 1002.
[71]. The authors’ first study in 2013 analyzed the reaction of defense attorneys to Daubert. See Jurs & DeVito, supra note 5, at 678-79.
[72]. Id. at 680, 701-02.
[73]. This point is at the heart of forum shopping. See, e.g., Christopher D. Cameron & Kevin R. Johnson, Death of a Salesman? Forum Shopping and Outcome Determination Under International Shoe, 28 U.C. Davis L. Rev. 769, 776-77 (1995) (noting that attorneys will file where they think they have the greatest chance of success).
[74]. When defense attorneys have a choice between Daubert and Frye venues, they use their removal power to choose the Daubert venue. Jurs & DeVito, supra note 5, at 680, 699-700. Defense attorneys file in the Daubert venues because they believe it increases the chances they can win by excluding expert testimony; however, plaintiffs’ attorneys file in non-Daubert venues, when possible, because they believe it increases the chances they can win by not excluding expert testimony. See Cameron & Johnson, supra note 73, at 776-77.
[75]. The concept of a “gatekeeping gradient” is crucial to understanding the results in this article and the authors’ previous article. When an attorney has a choice to file in two different venues, the attorney’s choice rests on his or her beliefs about the probability of success in that venue. Many factors affect whether a case will succeed in a particular venue. When a factor plays out differently in each venue, the attorney filing the case will have to determine whether that factor raises or lowers the chances that the case will succeed in each of the venues. If a factor affects the chances of success, then a venue gradient has occurred. In essence, the attorney will feel pressure to file in one venue as opposed to the other because of the difference in factors. The archetype for venue gradients is diversity of jurisdiction. See, e.g., Pease v. Peck, 59 U.S. 595, 599 (1855) (recognizing that the choice of federal court over state court might arise out of the attorney’s concern that state citizens might receive preferential treatment as compared to citizens of other states). Venue gradients are also at the heart of forum shopping. See, e.g., Note, Forum Shopping Reconsidered, 103 Harv. L. Rev. 1677, 1677-80 (1990) (recognizing that parties will utilize procedural or substantive differences in the law of different forums to choose where to file in order to maximize the chances of success in the lawsuit). A gatekeeping gradient is simply one form of venue gradient—attorneys feel pressure to choose a venue based upon differing gatekeeping standards in forums available to the plaintiff.
[76]. See supra notes 54-55 and accompanying text (discussing the Frye standard).
[77]. See supra notes 54-55; see also Kumho Tire Co., Ltd. v. Carmichael, 526 U.S. 137, 147-48 (1999) (reviewing the holding in Daubert and noting that it altered the requirements for judges in applying the federal rules of evidence).
[78]. Other factors may affect where attorneys file their cases and, thus, destabilize the filing rates. This study uses a fixed-effects analysis to limit the effect of these unaccounted-for variables. See infra note 92 and accompanying text; see also Jurs & DeVito, supra note 5, at 720-23 (employing a fixed-effects analysis to reduce the impact of confounding variables).
[79]. The authors accumulated the data from a state database and a federal database of court filings. Both databases are available to the public at the Inter-university Consortium for Political and Social Research (ICPSR). The “State Database” comes from the National Center for State Courts; it contains the number of case filings in state court and includes the adult and total population by state and year. Nat’l Ctr. for State Courts, Inter-univ. Consortium for Political & Social Research, ICPSR 3990, State Court Statistics, 2002: [United States] 2 (2004) [hereinafter State Court Statistics, 2002] (on file with authors); Nat’l Ctr. for State Courts, Inter-univ. Consortium for Political & Social Research, ICPSR 9266, State Court Statistics, 1985–2001: [United States] ii (2003) [hereinafter State Court Statistics, 1985–2001] (“This data collection provides comparable measures of state appellate and trial court caseloads by type of case for the 50 states, the District of Columbia, and Puerto Rico.”) (on file with authors); Nat’l Ctr. for State Courts, Inter-univ. Consortium for Political & Social Research, ICPSR 9266, State Court Statistics, 1985–1995: [United States], Trial Court Data, 1988–1992 (1997) [hereinafter Trial Court Data, 1988–1992] (listing variables and codes for trial-court data from 1988 through 1992) (on file with authors). The “Federal Database” derives from the Federal Judicial Center and contains the number of case filings in federal district court by state and year. See Fed. Judicial Ctr., Inter-univ. Consortium for Political & Social Research, ICPSR 3415, Federal Court Cases: Integrated Data Base, 2011, Part 2: Civil Terminations, 2001 (2002) [hereinafter Civil Terminations, 2001] (listing variables and codes for district-court data) (on file with authors).
[80]. The year field contains a standard four-digit year from 1990 to 2002 (1990, 1991, . . . , 2001, 2002). The state identifier is a numeric field containing values from 1 to 13, with each number corresponding to a specific state (Alaska = 1; Arizona = 2; Connecticut = 3; Florida = 4; Indiana = 5; Kansas = 6; Michigan = 7; Minnesota = 8; Missouri = 9; New York = 10; North Carolina = 11; Tennessee = 12; and Washington = 13). The binary field identifying whether the entry is a filing in federal or state court is “0” if the entry corresponds to a state filing and “1” if the entry corresponds to a federal filing.
[81]. The authors constructed the database using thirteen states: Alaska, Arizona, Connecticut, Florida, Indiana, Kansas, Michigan, Minnesota, Missouri, New York, North Carolina, Tennessee, and Washington. The authors chose these states for two reasons. First, they represent the states for which the authors had complete federal- and state-filings data for the study period. For other states that would have been appropriate for the study, either large gaps existed in the state data or the state data appeared inaccurate. Second, these thirteen states were also the subjects of the authors’ previous removal study; therefore, the symmetry of using the same states provides added value in examining the filing rates.
[82]. In each file of state-court data, the National Center for State Courts has compiled comprehensive case-filing data within each state during the period in question. See State Court Statistics, 1985–2001, supra note 79. Since this analysis involves tort filings and later normalizes for population, the authors limited the data to only the following necessary variables: state (“STATEAB”); court type (“CRTTYPE”); state population (“TOTALPOP”); adult population (“ADULTPOP”); and tort cases filed (“TORTFILE”). See Trial Court Data, 1988–1992,supra note 79, at 7 (listing and explaining the variables for the dataset). The authors also limited the court type to courts of general jurisdiction, which handle civil tort claims that are the focus of this study (CRTTYPE = “1”). Using these parameters, each state would then have the available amount of tort suits filed in a specific year, as demonstrated by Kansas in 1997, where the variable “TORTFILE” contains 6194 tort filings.
[83]. The Federal Database includes a wide range of cases over an extended period of time. As a result, it was necessary to limit the data to just those filings that were relevant to our inquiry. The Federal Database contains a set of codebooks explaining what fields the data includes and what those fields mean. See Fed. Judicial Ctr., Inter-univ. Consortium for Political & Social Research, ICPSR 8429, Federal Court Cases: Integrated Data Base, 1970–2000, Part 117: Civil Terminations, 2000 (2001) (covering the year 2000); Fed. Judicial Ctr., Inter-univ. Consortium for Political & Social Research, ICPSR 8429, Federal Court Cases: Integrated Data Base, 1970–2000, Part 116: Civil Terminations, 1999 (2001) (covering the year 1999); Fed. Judicial Ctr., Inter-univ. Consortium for Political & Social Research, ICPSR 8429, Federal Court Cases: Integrated Data Base, 1970–2000, Part 115: Civil Terminations, 1998 (2001) (covering the year 1998); Fed. Judicial Ctr., Inter-univ. Consortium for Political and Social Research, ICPSR 8429, Federal Court Cases: Integrated Data Base, 1970–2000, Part 104: Civil Terminations, 1997 (2001) (covering the year 1997); Fed. Judicial Ctr., Inter-univ. Consortium for Political & Social Research, ICPSR 8429, Federal Court Cases: Integrated Data Base, 1970–2000, Part 103: Civil Terminations, 1996 (2001) (covering the year 1996); Fed. Judicial Ctr., Inter-univ. Consortium for Political & Social Research, ICPSR 8429, Federal Court Cases: Integrated Data Base, 1970–1995 (1997) (covering the years 1988 through 1991); Fed. Judicial Ctr., Inter-univ. Consortium for Political & Social Research, ICPSR 8429, Federal Court Cases: Integrated Data Base, 1970–1994 (1997) (covering the years 1988 through 1991). All codebooks are on file with the authors.
[84]. The authors restricted the data to only those cases: (1) identified as torts; (2) in the states subject to the study; (3) filed in district court; and (4) over which the district court had jurisdiction because of diversity of citizenship.
[85]. See supra note 82 (explaining how the authors limited the total National Center for State Court’s database to only the necessary variables).
[86]. For example, a federal-filing entry for 1997 in Kansas would contain three values: “1997”; “6”; and “1.” An entry corresponding to a state filing would contain the following values: “1997”; “6”; and “0.”
[87]. Therefore, one dataset covers the creation of the gatekeeping gradient while the other covers the destruction of that gradient.
[88]. Alaska, Arizona, Connecticut, Florida, Indiana, Kansas, Michigan, Minnesota, Missouri, New York, North Carolina, Tennessee, and Washington.
[89]. The years 1990 through 2000.
[90]. See Jurs & DeVito, supra note 5, at 698.
[91]. The following states did not adopt Daubert during this time: Arizona, Florida, Kansas, Michigan, Minnesota, Missouri, New York, and Washington. Cheng & Yoon, supra note 6, at 493 fig.7. Alaska adopted Daubert in 1999; Connecticut adopted it in 1997; Indiana adopted it in 1995; North Carolina adopted it in 1995; and Tennessee adopted it in 1997. Id.; see generally Alice B. Lustre, Annotation, Post-Daubert Standards for Admissibility of Scientific and Other Expert Evidence in State Courts, 90 A.L.R.5th 453 (2001) (discussing the cases where the courts applied Daubert or some other admissibility test).
[92]. This exclusion is a vital step. Keeping the data for the period after a state has adopted Daubert would mean keeping data that has a different slope. For example, tacking Figure 2 onto the end of Figure 1 and then analyzing that data would treat the slope like it was essentially flat—masking the very different effects at different times. To prevent this faulty analysis, this study requires isolating the Figure 1-type data from the Figure 2-type data.
[93]. Ideally, in comparing data from two different subjects, the authors assume that the statistical properties of the data for each subject are, overall, the same for the other subject. When this occurs, homogeneity exists. But when homogeneity is lacking, heterogeneity exists. In essence, unobserved relevant variables may exist that are correlated with the observed variables, and the value of those variables may differ from subject to subject. A heterogeneous population has unique or individual characteristics as compared to another population. See Damodar N. Gujarati & Dawn C. Porter, Essentials of Econometrics 282, 282-83 (Brent Gordon et al. eds., 4th ed. 2010).
[94]. The years are 1990, 1991, 1992, 1994, 1995, and 1996.
[95]. Alaska, Arizona, Connecticut, Florida, Indiana, Kansas, Michigan, Minnesota, Missouri, New York, North Carolina, Tennessee, and Washington.
[96]. The years 1994 through 2002.
[97]. See Kaye, Bernstein & Mnookin, supra note 2, § 6.4.2, at 226 (regarding the issue of states choosing between the Daubert standard and the Frye standard); Lustre, supra note 91, at 453 (listing states and their current admissibility standards for scientific evidence). In the period of this study (1990–2002), the following states did not adopt Daubert: Arizona, Florida, Kansas, Michigan, Minnesota, Missouri, New York, and Washington. Cheng & Yoon, supra note 6, at 493 fig.7 (showing date of adoption of Daubert for these states during the period 1990–2000); Lustre, supra note 91, at 453.
[98]. For example, for Connecticut, which adopted Daubert in 1997, the authors retained data relating to 1994–1996 and 1998–2000. See Cheng & Yoon, supra note 6, at 493 fig.7.
[99]. The authors created the binary field “Federal_Adopt_Daubert.” For each entry in the dataset, the authors set this field to “0” for all entries that had years less than 1993 and “1” for all entries that had years greater than 1993. As noted above, the authors eliminated the year 1993 from the dataset; therefore, this field did not need to have a value assigned to it when the year was 1993. See supra note 89-90 and accompanying text.
[100]. All measures of statistical significance discussed in this article relate to the p-value of a statistical hypothesis. This study considers a result to be statistically significant if its corresponding p-value is less than or equal to 0.05. Therefore, only a one-in-twenty chance (or 5% chance) exists that this study’s result is due to chance. See David A. Hensher, John M. Rose & William H. Greene, Applied choice Analysis: A Primer 47 (2005). In addition, using a p-value of 0.05 or less as a basis for statistical significance is consistent with general practice. See, e.g., id.; Scott E. Maxwell & Harold D. Delaney, Designing Experiments and Analyzing Data: A Model Comparison Perspective § 2.8.3 (2d ed. 2004).
[101]. The data selection in this article’s study only yields two kinds of entries: (1) cases filed when both federal and state courts used Frye; and (2) cases filed when the federal court used Daubert while the state court continued to use Frye.
[102]. This division required creating the binary field “State_Adopt_Daubert.” For each entry in the dataset, the authors set the field to “0” when the state, in that year, had not adopted Daubert and “1” when the state, in that year, had adopted Daubert as the gatekeeping standard.
[103]. The authors recognize that correlation does not equal causation. At the same time, a strong statistical correlation consistent with an underlying theory supports that theory.
[104]. This study uses a logistic as opposed to linear regression in the fixed-effects analysis because researchers generally prefer a logistic when the independent variable is categorical/binary. See, e.g., Alan Agresti & Christine Franklin, Statistics: The Art and Science of Learning From Data 610 (2007); Gujarati & Porter, supra note 93, at 387-89.
[105]. The authors’ previous study provides a more detailed description of a fixed-effects analysis. See Jurs & DeVito, supra note 5, at 718-21.
[106]. Dummy variables are variables that you create that have a value of “1” if a condition is met and a value of “0” otherwise. Damodar Gujarati, Econometrics by Example 47 (2011). This study uses one dummy variable for each year and for each state. For example, the dummy variables would include: “Dummy_Alaska”; “Dummy_Arizona”; etc., and “Dummy_1990”; “Dummy_1991”; etc. The addition of the dummy variables changes our data from having three variables to having twenty-nine. The variables still include the filing year, filing state, and federal filing, but now they include one for each state (thirteen dummy state variables) and one for each year (eleven or thirteen dummy state variables, depending on the year range). A dummy variable is set to “true” if it is true of that entry, but it is set to “0” if otherwise. For example, assuming the dummy state variables came first, the entry for cases in Kansas filed in 1997 in federal court would change from “<1997, 06, 1>” to “<1997, 06, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0>,” with the first underlined “1” corresponding to “Dummy_Kansas” and the second underlined “1” corresponding to “Dummy_Year.”
[107]. The “fixed” in the fixed-effects model corresponds to the idea that the intercept for each individual (in our case, states and years are individuals) is time invariant. This study then uses the intercept for the first individual as the benchmark intercept. The dummy variables “will show by how much the intercept coefficient of the individual that is assigned a dummy variable will differ from the benchmark category.” Gujarati, supra note 106, at 283.
[108]. See infra Appendix.
[109]. When performing the regression analysis using the formula, one must take care to avoid a problem called the “dummy variable trap.” See Gujarati, supra note 106, at 283. In this study, because only one dummy variable exists for each state and one for each year, the study encountered perfect colinearity between (at least) one state dummy variable and one year variable and the rate-of-removal intercept. See id. at 48. To avoid this problem, the study dropped (at least) one state dummy variable and one year dummy variable when performing the regression analysis.
[110]. The correlation coefficient for the federal filings variable has a p-value of less than 0.0005, meeting the test for statistical significance—which requires the p-value to be less than or equal to 0.05.
[111]. The data was normalized by calculating a set of weights equal to the total population (adult and non-adult) in a given state in 1990, divided by the population in that state in each subsequent year and then multiplying the number of torts filed in a given year in that state by the related weight.
[112]. The correlation coefficient for the federal-filings variable has a p-value of less than 0.0005, meeting the test for statistical significance, which requires the p-value to be less than or equal to 0.05.
[113]. The correlation coefficient for the federal-filings variable has a p-value of less than 0.0005, meeting the test for statistical significance.
[114]. This normalization accounts for total population.
[115]. The correlation coefficient for the federal-filings variable has a p-value of 0.001, meeting the test for statistical significance.