please help me to understand the porter analysis here, include quotes and citations which would help bolster this as I am using it in an essay: B. PORTER FACTORS
The state must demonstrate that the expert's methods for reaching her conclusions are reliable. The court will next turn to a discussion of the Porter factors that are to be considered.



1. General Acceptance in the Relevant Scientific Community
In this case, the defendant does not appear to argue that the AFTE methodology lacks general acceptance in the relevant field, but rather [*14]  points to certain skeptics of the methodology and the reasons for their skepticism. Indeed, Abruquah, broadly relied on by the defendant for many of his claims, itself held that "we do not question that firearms identification is generally reliable, and can be helpful to a jury, in identifying whether patterns and markings on 'unknown' bullets or cartridges are consistent or inconsistent with those on bullets or cartridges known to have been fired from a particular firearm," and that "studies and other information ... support the use of the AFTE Theory ...." Abruquah v. State, supra, 483 Md. 694-96. Indeed, Terrell, which was cited by our Supreme Court in State v. Raynor, supra, 327 Conn. 548 n.13, concluded that the state had established that "the basic techniques employed by firearm and [toolmark] examiners are generally accepted in the relevant scientific community." State v. Terrell, supra, Superior Court, DocketNo. CR-17-0179563; see also United States v. Romero-Lobato, 379 F. Supp. 3d 1111, 1122 (D. Nev. 2019) (collecting cases finding AFTE theory to be widely accepted in relevant community and finding same).

Based on the foregoing, and the testimony of Beninati describing the methodology and worldwide recognition of such methodology by the American Academy of Forensic Sciences (a multi-disciplinary organization which works globally) [*15]  and the International Association for Identification, the court finds that the AFTE method easily satisfies the first Porter factor.



2. Testing and Peer Review of the Methodology
The state must next show that the methodology underlying the scientific evidence has been tested and subjected to peer review.

The defendant argues that the Supreme Court of Maryland in Abruquah, found that the evidence was "mixed" as to whether the methodology had been peer reviewed. The Abruquah court described the findings of two reports of experts "outside of the firearms and toolmark identification field" that were critical of the AFTE theory. Abruquah v. State, supra, 483 Md. 662-63. One report was authored by the National Research Council of the National Academies of Science (NRC) in 2009. Included in NRC's findings was a concern that "[t]he validity of the fundamental reproducibility of firearms-related toolmarks has not yet been fully demonstrated." (Internal quotation marks omitted.) Id., 664. That court also referenced a report from the 2016 President's Council of Advisors on Science and Technology (PCAST), which "focused on whether there had been a demonstration of both 'foundational validity' and 'validity as applied.'" Id., 665. "Foundational validity, [*16]  according to PCAST, requires that the method 'be shown, based on empirical studies, to be repeatable, reproducible, and accurate, at levels that have been measured and are appropriate to the intended application.' Validity as applied requires 'that the method has been reliably applied in practice,'" (Emphasis in original.) Id.

PCAST asserted that appropriate studies had not been conducted and were not properly designed, and also noted that there was a dearth of "black box" studies with respect to the methodology.

"A black box study assesses the accuracy of examiners' conclusions without considering how the conclusions were reached. The examiner is treated as a 'black-box' and the researcher measures how the output of the 'black-box' (examiner's conclusion) varies depending on the input (the test specimens presented for analysis). To test examiner accuracy, the 'ground truth' regarding the type or source of the test specimens must be known with certainty." Organization of Scientific Area Committees for Forensic Science, "OSAC Draft Guidance on Testing the Performance of Forensic Examiners," (2018), available at https://www.nist.gov/document/drafthfeguidancedocument-may8pdf (last visited [*17]  April 12, 2024).

"The sole exception to PCAST's negative critique of study designs was a study performed by the United States Department of Energy's Ames Laboratory (the 'Ames I Study'), which PCAST called 'the first appropriately designed black-box study of firearms [identification]." Abruquah v. State, supra, 483 Md. 666.

"Nonetheless, PCAST observed that that study . . . was not published in a scientific journal, had not been subjected to peer review, and stood alone. . . . PCAST therefore concluded that 'firearms analysis currently falls short of the criteria for foundational validity' and called for additional testing," (Citation omitted.) Id.

Beninati described the steps that have been taken in the field since the publication of the PCAST report. Other courts have documented the landscape since the PCAST report as well. Parenthetically, the authors of the PCAST report themselves have subsequently addressed its contents. Other courts have pointed out that the authors of the PCAST report "explicitly acknowledged that the rigorous, entirely objective, infallible, and certain standards that prevail within a scientific discipline are not necessarily properly applied in a courtroom. Indeed, the authors were careful to emphasize [*18]  that their conclusions should not be read as commentary on the admissibility of toolmark evidence in courts." United States v. Johnson, United States District Court, DocketNo. (S5)-16-CR.-281 (PUG), 2019 U.S. Dist. LEXIS 39590, 2019 WL 1130258, *14 (S.D.N.Y. March 11, 2019), aff'd, 861 F. App'x 483 (2d Cir. 2021), cert. denied, 142 S. Ct. 1219, 212 L. Ed. 2d 221 (2022). The PCAST report, in fact, specifically states: "[w]hether firearms analysis should be deemed admissible based on current evidence is a decision that belongs to the courts." (Internal quotation marks omitted.) Id.

According to the Johnson court, participants in the scientific studies have actually testified that the reports were not intended to suggest that toolmark identification or ballistics analysis is unreliable for purposes of expert evidence in court. "Notably, the co-chair of the committee that issued the [2009 NRC report], Judge Harry T. Edwards, has specifically noted that the [2009 NRC] report is not a law reform proposal and that 'whether forensic evidence in a particular case is admissible under applicable law is not coterminous with the question whether there are studies confirming the scientific validity and reliability of a forensic science discipline.' . . . [T]he United States has produced the sworn statement of the Chairman of the group that produced the [2008 NRC report, Dr. Rolph]. In that statement, Dr. Rolph states that the purpose [*19]  of the 2008 N[RC] report was not to pass judgment on the admissibility of ballistics evidence in legal proceedings. . . . In fact, the question of legal admissibility 'was explicitly ruled out of [his committee's] charge.' . . . [H]is committee did not actually evaluate the fundamental assumptions of firearms and toolmark identification that underlay many courts' allowance of ballistics and firearm expert testimony.'" (Citations omitted; internal quotation marks omitted.) Id., *14 n.7.

The court in United States v. Harris, 502 F. Supp. 3d 28, 40 (D.D.C. 2020) observed that "much of the literature in this discipline has been published in the AFTE Journal, a peer reviewed journal that 'publishes articles, studies and reports concerning firearm and toolmark evidence.' . . . The AFTE Journal uses a formal process for article submissions, including 'specific instructions for writing and submitting manuscripts, assignment of manuscripts to other experts within the scientific community for a technical review, returning of manuscripts to other experts within the scientific community for clarification or re-write, and a final review by the Editorial Committee.'" (Citation omitted.)

The Harris court also referenced a number of courts that concluded that [*20]  publication in the AFTE Journal satisfied this prong of the Porterl Daubert analysis. See, e.g., United States v. Romero-Lobato, supra, 379 F. Supp. 3d 1118; United States v. Johnson, 2019 U.S. Dist. LEXIS 39590, 2019 WL 1130258, *16; United States v. Ashburn, 88 F. Supp. 3d 239, 245-46 (E.D.N.Y. 2015); United States v. Taylor, 663 F. Supp. 2d 1170, 1176 (D.N.M. 2009); United States v. Monteiro, 407 F. Supp. 2d 351, 366-67 (D. Mass. 2006).

Still, the concerns or recommendations raised by the PCAST report have been subsequently addressed by experts in the field of toolmark and firearm analysis. Beninati testified that she was familiar with both the 2009 NRC report and the 2016 PCAST report and the recommendations made therein. According to Beninati, since 2016, twenty-one (21) additional studies have been conducted and peer reviewed. Those studies have supported the methodology employed by the State Lab and others. Specifically, with respect to those studies, peer review and publication have occurred in both forensic2 and non-forensic3 journals and have been accepted by the relevant scientific community. "The mere fact that an expert's opinion is derived from subjective methodology does not render it unreliable." United States v. Romero-Lobato, supra, 379 F. Supp. 3d 1120. Forensic firearm and toolmark analysis results of the State Lab are subject to review by a second examiner who independently may verify or disagree with the original result. From there the results are subject to yet another, independent, administrative review. Because the firearm and toolmark identification [*21]  methodology used by Beninati has been subject to peer review and publication as described above, the court finds this Porter factor to also weigh in favor of admission.



3. The Known and Potential Rate of Error
The state must also show the known or potential rate of error of the methodology. The defendant contends that the Abruquah court did not find a reliable error rate, and cites both the amicus analysis and the article by Alan Dorfman on this point. Dorfman also references a study known as Ames 11.4 Mirroring the discussion in the Dorfman article, the defendant focuses on the impact of "inconclusive" findings on error rate. According to Dorfman, in general terms, error rates that have been published in recent black box studies of forensic firearms examiner performance have been very low, typically below one percent. 'These low error rates have been challenged, however, as not properly taking into account one of the categories, "inconclusive," that examiners can reach in comparing a pair of bullets or cartridges. These challenges have themselves been challenged; how to consider the inconclusives and their effect on error rates is currently a matter of sharp debate.

It is Dorfman's contention, and that of the defendant, that as a result, from all these perspectives, it is impossible to simply read out trustworthy estimates of error rates from those studies which have been carried out to date. At most, one can put reasonable bounds on the potential error rates. According to this argument, these are much larger than the nominal rates reported in the studies.

The defendant asserts that to get straightforward, sound estimates of error rates requires a challenging but critical improvement to the design of firearms studies. According to Dorfman, a "proper study" — one in which inconclusives are not potential errors, and which yields direct, sound estimates of error rates — will require new objective measures or blind proficiency testing embedded in ordinary casework. A. Dorfman & R. Valliant, supra, 5 Forensic Sci. International: Synergy, 1.

Other courts have addressed the issue of error rates. In United States v. Lee, United States District Court, Docket, 2022 U.S. Dist. LEXIS 150054, 2022 WL 3586164, *3 (N.D. Ill. 2022), the court referenced a study, also discussed by Beninati, by the Collaborative Testing Service. The Lee court observed that "data completed by the Collaborative Testing Service shows error rates [*23]  between 0.9% and 1.5% in conclusions about whether particular casings or bullets were fired by particular guns." Id.; see also United States v. Brown, 973 F.3d 667, 704 (7th Cir. 2020) ("[a]lthough the error rate of this method varies slightly from study to study, overall, it is low — in the single digits . . . sometimes better than algorithms developed by scientists"), cert. denied, 141 S. Ct. 1253, 208 L. Ed. 2d 638 (2021).

The court in United States v. Rhodes, United States District Court, DocketNo. 3:19-cr-00333 (MC), 2023 U.S. Dist. LEXIS 7528, 2023 WL 196174 (D. Or. January 17, 2023) addressed the potential impact of error rates to the Daubers-Porter analysis, with particular attention to the issue of "inconclusive" findings. As the defendant herein argues, the defendant in Rhodes argued that the error rate of toolmark analysis weighs against admission because it does not count inconclusive results as an error.

In its analysis, the Rhodes court stated "[it] considers whether the technique has a known or potential rate of error. . . . An error can either be a false negative — an incorrect finding of dissimilarity -- or a false positive — an incorrect affirmative identification. . . . In the context of firearm and toolmark analysis, then, a false negative means that a certain bullet came from the tested firearm, but the examiner concludes that it did not; conversely, a false positive means that a certain bullet did not come from [*24]  the tested firearm, but the examiner concludes that it did. [T]he focal point of the inquiry should be on the rate of false positives, as this is the type of error that could lead to a conviction premised on faulty evidence.... [T]he weight of authority suggests the potential error rate [for toolmark analysis] is between 0-1%." (Citations omitted; emphasis in original; internal quotation marks omitted.) Id. 2023 U.S. Dist. LEXIS 7528, *10-*11, [WL], *3-*4.

The Rhodes court further observed, when discussing the evidence with respect to error rates, that "no matter the study design, no matter what the study conditions are, the false-positive rate is consistently around [1%] or less . . . [b]ut even a higher error rate of error is still well short of the [5%] error rate that the PCAST report identified as an acceptable error rate from a scientific perspective. (Citations omitted; internal quotation marks omitted.) Id. 2023 U.S. Dist. LEXIS 7528, *11, [WL], *4; see also United States v. Cloud, 576, F. Supp. 3d 827, 843 (2021) (discussing error rate studies).

Beninati discussed the error rate prong and the alleged impact of inconclusive results. She described the concern that inconclusive results are not properly accounted for as not recognizing the requirement that inconclusive decisions are decisions that must meet certain criteria and [*25]  are to be used when there is insufficient data upon which to base a more conclusive opinion. According to Beninati, one premise of the concern about the potential impact of inconclusives on error rate calculations is the notion that because examiners know they are in a study that could impact the validity of the discipline, they are more likely to use the inconclusive result much more frequently because it is considered neither correct or incorrect. She also indicated criticism existed that because of the alleged 'pristine condition' of the samples in error rate studies, inconclusive results should be considered errors. However, she pointed out that several studies used confiscated firearms that were in circulation in the general population and some used damaged specimens in the samples, neither of which led to significant error rates. In addition, not all firearms in the studies produced clear and reproducible marks.

Beninati cited to a number of studies, to wit: J. Smith, Beretta Barrel Fired Bullet Validation Study, 66 (2) J. Forensic Sci. 547 (2021); C. Chapnick et al., Results of the 3D Virtual Comparison Microscopy Error Rate (VCMER) Study for Firearm Forensics, 66 (2) J. Forensic [*26]  Sci. 557 (2021); E. Law & K. Morris, Evaluating Firearm Examiner Conclusion Variability Using Cartridge Case Reproductions, 66 (5) J. Forensic Sci. 1704 (2021); J. Hamby et al., A Worldwide Study of Bullets Fired from 10 Consecutively Rifled 9MM Ruger Pistol Barrels-Analysis of Examiner Error Rate, 64 (2) J. Forensic Sci. 551 (2019); W. Kerkhoff et al., A Part-declared Blind Testing Program in Firearms Examination, 58 (4) Sci. & Just. 258 (2018); M. Keisler et al., Isolated Pairs Research Study, 50 AFTE J. 56 (2018); T. Smith et al., A Validation Study of Bullet and Cartridge Case Comparisons Using Samples Representative of Actual Casework, 61 (4) J. Forensic Sci. 939 (2016); and S. Bajic et al., U.S. Dept. of Energy (Ames II), which contained error rate conclusions. Each of the studies she referenced have error rates within the parameters discussed by the Rhodes court. This court agrees with the approach of the court in Rhodes with respect to the impact of inconclusive results on the reliability of error rates. As stated in its opinion:

[The court] finds that while the relatively high rate of inconclusive results may be relevant to certain policy determinations — such as failing to eliminate a suspect [*27]  from consideration — those concerns do not relate to the issue of paramount importance to trial courts; i.e., the false positive rate which may result in a wrongful conviction. [E]vidence of the false negative rate is often equivocal. While it might suggest a generally error-prone method, it is equally consistent with a very conservative method with a low false positive error rate. That is, a method may be designed to lower its false positive error rate by accepting a large number of false negatives out of an abundance of caution. . . . [The opinion] that inconclusive results should be viewed as false positive errors (i.e., included among false identifications) . . . fails to make logical sense: while under laboratory conditions such inconclusives [may be] some type of error, it does not follow that inconclusives are functionally the same as a false conclusion by an examiner who attributes a cartridge casing to a gun that did not fire it. . . . Put another way, while an inconclusive result is an error insofar as it means the methodology did not produce an answer, it is not an error in the sense that it falsely attributes a cartridge or casing to the wrong firearm.

(Citations omitted; [*28]  internal quotation marks omitted.) United States v. Rhodes, 2023 U.S. Dist. LEXIS 7528, 2023 WL 196174, *4.
In United States v. Felix, United States District Court, 77 V.I. 714, 2022 WL 17250458 (D.V.I. 2022), another case cited by the defendant, the court observed during its analysis of the impact of inconclusive findings on error rates that:

counting inconclusive responses as errors is not a satisfactory solution because counting them as such would likely artificially inflate the error rate by counting two responses as incorrect and only one response as correct. Moreover, practical considerations also counsel against counting inconclusive responses as errors. As the PCAST Report correctly observes, if an examiner arrived at an inconclusive determination in casework, the evidence is unlikely to be used against a defendant in court. . . . As such, counting inconclusive responses as errors has the effect of improperly equating inconclusive responses with the false positive or false negative errors that could adversely affect judicial proceedings by misleading the factfinder.

(Citation omitted; emphasis in original.) Id. 77 V.I. 714, 742, [WL], *16.
The court does not find the defendant's argument regarding the impact of inconclusive results to be persuasive. Even accepting the higher error rates calculated by other courts, the error rate for toolmark analysis weighs in the state's [*29]  favor.

4. The Prestige and Background of the Expert Witness Supporting the Evidence
The state presented the testimony of Beninati in support of the methodology of firearm and toolmark analysis. Beninati has been employed by the State Lab since 2020. She holds a bachelor's of science degree in criminal justice from the University of North Alabama. Prior to joining the State Lab, she was employed for a company that was contracted to do work for the Bureau of Alcohol, Tobacco, and Firearms (ATF). While at ATF, she worked with the National Integrated Ballistics Information Network (NIBIN), entering cartridge cases into a computer system that took high-definition photographs of areas of interest on cartridge cases which then used an algorithm to populate a list for comparisons on the computer. Since her employment at the State Lab, she has received cartridge case and bullet analysis training through the State Lab itself. The training included assembling and disassembling firearms of all types, completing tests, comparative analysis training, and practical exercises. The comparative analysis training involved comparing bullets and cartridge cases to test fires to determine if they were fired [*30]  from a specific firearm. The comparative analysis training was conducted by the Nichols Forensic Science Consulting Firearm and Toolmark Examiner Training Academy ("Academy"), which she attended in 2021. The Academy was six to eight months in duration. The Academy required four hundred hours of time using a comparison microscope looking at bullets known to have been fired from the same or a different gun. She was required to take both written exams and practical exercises.

Following the training at the Academy, Beninati was required to take a competency test pertaining to comparative analysis and was certified by the State Lab for case work. She has been conducting comparative analysis at the State Lab since the end of 2021 and estimated that she has performed over one hundred comparisons since that time, all of which were then subject to an independent verification process by a second examiner, and an additional administrative review thereafter.

The State Lab has standard operating procedures generally for the lab, and specific standard operating procedures for firearm and toolmark analysis. The State Lab is accredited, meaning its procedures are recognized and it is ensured that their [*31]  procedures are being followed. There are external and internal audits. Beninati has been both an initial and secondary analyst. The State Lab has instituted checks and balances to ensure that comparisons are performed to AFTE standards.

Beninati described the use of a light comparison microscope during the analysis phase of the process. An analyst is able to take photographs while using the microscope and can then annotate the photographs to demonstrate certain findings.

Including the training Beninati received at the Academy, she has been involved in eight to nine hundred comparisons of ballistic evidence. She has been previously qualified as an expert in eight prior cases, and is required to undergo yearly proficiency tests administered by an outside agency, with the results forwarded to the State Lab. There was nothing in the evidence developed that raised any questions or concerns regarding Beninati's competence in her field, or her level of expertise.

The court concludes that Beninati is qualified to testify as an expert in the field of firearm and toolmark identification, and thus the state has satisfied this prong of the test.

5. The Extent to Which the Technique at Issue Relies [*32]  on Subjective Judgments Made by the Expert Rather than on Objectively Verifiable Criteria
As noted in the 2023 commentary to Rule 702 of the Federal Rules of Evidence, some types of expert testimony will be more objectively verifiable than others. "The fact that there are subjective elements to the firearm and toolmark methodology is not enough to show that the theory is not 'testable.' Indeed, studies have shown that the 'AFTE theory is testable on the basis of achieving consistent and accurate results.'" United States v. Harris, supra, 502 F. Supp. 3d 37; accord United States v. Rhodes, 2023 U.S. Dist. LEXIS 7528, 2023 WL 196174, *2-*3.

The defendant argues that although "examiners personally look at the cartridges or projectiles, and subjectively compare their markings," there "is no computer program or objective mathematical analysis that determines whether they are consistent." However, the mere fact that these are subjective elements to the firearm and toolmark identification methodology is not enough to show that the theory is not "testable." United States v. Harris, supra, 502 F. Supp. 3d 37. In support of that conclusion, the Harris court cited a study, P. Duez et al., Development and Validation of Virtual Examination Tool for Firearm Forensics, 63 J. Forensic Sci. 1069 (2018) (Heat Map Study). According to Harris:

"[t]he Heat Map Study indicated that firearm examiners from fifteen different laboratories, all conducting an independent assessment, were 'mostly using the same amount and same location of microscopic [*33]  marks when concluding identification.' . . . Critically, the trained examiners also correctly reported 100% of known matches while reporting no false positives or false negatives."

(Citation omitted.) United States v. Harris, supra, 502 F. Supp. 3d 37.
The Harris court also addressed recent scientific studies that meet the PCAST black box model requirements and demonstrate the reliability of the firearm and toolmark identification method:

[A]nother recent black box study test[ed] the identification of fired casings, which resulted in a .433% false positive error rate from three errors among 693 total comparisons. See [C. Chapnick et al., supra, 66 (2) J. Forensic Sci. 557]. A third post-PCAST Report study also followed the PCAST recommended black-box model and found that of 1512 possible identifications tested, firearms examiners correctly identified 1508 casings to the firearm from which the casing was fired. [M. Keisler el al., supra, 50 AFTE J. 58]. This evidence indicates that even under the PCAST's stringent black-box only criteria, firearm and toolmark identification can be tested and reasonably assessed for reliability.

(Citation omitted; emphasis omitted.) United States v. Harris, supra, 502 F. Supp. 3d 38; see also United States v. Hunt, 464 F. Supp 3d 1252, 1257 (W.D. Okla. 2020) (collecting cases addressing testability), aff d, 63 F.4th 1229 (10th Cir. 2023).
Beninati discussed empirical and machine based studies [*34]  with respect to firearm and toolmark identification. Empirical studies are those studies detailing research that is based on observation and measurement of phenomena, as directly experienced by the researcher. Machine based studies are those studies in which toolmarks have been acquired using instrumental means, are compared, and then analyzed using algorithms designed by researchers to assess the similarity of toolmark profiles and/or topographies that have been collected. The data is then collected and then analyzed both quantitatively and objectively in the overall assessment of toolmark similarly or dissimilarly. According to Beninati, machine based studies include; Welet et al. (2021); Zhang et al. (2020); Roberge et al, (2019); Lilien et al. (2019); Song (2017); and Spotts & Chumbley (2015).

Beninati stated that both forms of validation studies support the methodology used by AFTE examiners. The court therefore finds that this Porter factor also weighs in favor of admission.



6. Whether the Expert can Present and Explain the Data and Methodology in a Manner that Assists the Jury
The state has established that Beninati is able to present and explain the data and methodology in a manner [*35]  that assists the jury, and the court finds that she has reliably applied the principles and methods of AFTE, as described in her testimony, to the facts of this case.



7. Whether the Technique was Developed Solely for Purposes of Litigation
The tool used to analyze ballistic evidence, to wit: the comparison microscope, was originally developed for purposes unrelated to firearm and toolmark identification. Further, this field of study is available in college curriculums throughout the United States.



CONCLUSION
In conclusion, after a careful review of Beninati's expert testimony, and review of relevant case law addressing the question of the sufficiency of the methodology of firearm and toolmark analysis within the framework of Porter, encompassing analysis of said methodology in decisions promulgated after Terrell and including Raynor and decisions contemporaneous with and subsequent to Raynor, that the state has met its relevance and reliability burdens and has established that the testimony at issue is in fact derived from and based on a scientifically reliable methodology.

The court finds the AFTE methodology testable, subject to peer review and publication, supported by a low error rate [*36]  and generally accepted in the relevant field. Accordingly, the court considers the presentation of the evidence here to be more probative than prejudicial.

The defendant's contentions are issues that can be raised on cross-examination, as this argument goes to the weight of the evidence, not its admissibility.

State v. Raynor, Docket No. HHD-CR13-0667367, 2024 Conn. Super. LEXIS 1520, at *13-36 (Super. Apr. 16, 2024)