Appeal regarding Public Comment on ISO 17020

Craig Beyler, Chair OSAC Subcommittee on Fire and Explosion Investigation and member of the Virtual 

Committee that approved 17020

16 February 2017

Comment Submitted: 

17020 was developed for bodies inspecting to determine compliance with expectations. This is not 

useful for use in forensic investigations in which we seek to avoid any expectations. There is potential 

value in a standard for forensic investigation, but sadly this document does not address this. 17020 does 

not address the development of findings from an inspection. The work product is simply a listing of 

variations from the expectation.

Subcommittee Chair Response: 

Regarding the ISO/IEC-17020 standard for forensic agencies, the term Inspection applies to crime scene 

investigations and/or examination of forensic evidence. The term Inspector also applies to an 

investigator who uses professional judgment to examine a crime scene with the aim to contribute to 

determining what, where, when, how, and why something happened and who was involved. Of those 

US‐based accrediting bodies that offer accreditation to forensic organizations, two offer accreditation 

and have accredited forensic organizations to ISO/IEC 17020. ISO/IEC 17020 is referenced in the 

International Laboratory Accreditation Cooperation (ILAC) Guidance document, ILAC G19:08/2014 

Modules in a Forensic Science Process., http://ilac.org/?ddownload=805 which is used globally by 

Accrediting Bodies around the world. ISO/IEC 17020 addresses the crime scene/examination process 

and while it does not further address the analysis of the findings, that does not make the document nonrelevant nor not useful in its application to crime scene investigations. It just speaks for the need for 

another standard to be developed that addresses this stage of the investigation. As such, the comment 

received is considered non-persuasive and the standard should be posted on the registry.

Rationale for the Appeal:

ISO 17020 is a conformity assessment standard for bodies performing inspections. Forensic science is in 

no way conformity assessment. The accreditation community via ILAC has taken it upon themselves to 

make decisions for the forensic science community with respect to the appropriate basis for forensic 

science body accreditations (ILAC-G19:08/2014). While I can accept that they did this with all good 

intentions, they have force fit us into a conformity assessment standard that is not consistent with our 

activities. It is time for us to assert ourselves and define standards for ourselves and insist that the 

accreditation community follow our lead.

The virtual subcommittee to move ISO 17020 and ISO 17025 through the registry process did not do 

appropriate due diligence in their consideration of the standards. As my comments as a member of that 

subcommittee reflect, the subcommittee did not take the time to fully explore and question the 

standards and their appropriateness for our purposes. A single phone conference was convened on 

short notice only days before the vote deadline. This was inadequate and ill-considered. Comments 

were dispositioned by the chair alone, rather than by the subcommittee. I do not question the good 

intentions of the chair, but I do take issue with the process. My public comment, similarly was voted 

upon by the virtual committee with no meeting. The response was formulated by the chair and 

summarily put to a vote without any form of discussion or consideration. The entirety of the processing 
for inclusion in the OSAC Registry has evaded the processes set by OSAC to assure due consideration. 

The QIC should examine the process employed and determine if process changes are indicated. 

The subcommittee did not participate in the development of the registry forms. We did not vote on the 

forms. We had only a single meeting in the entire history of the subcommittee and that was done at the 

very last minute in response to my objections. The response to my concerns for process were pro forma 

at best and could be regarded as dismissive. OSAC is its processes and this virtual committee has shown 

that the processes at OSAC are broken. There had been NO oversight of the operations of the virtual 

committee. The QIC and FSSB have been surprised by my commentary on the lack of process. No one 

was watching and no one was checking. As a result, the processes were not followed. ISO 17020 should 

be rejected for inclusion in the registry at this time if for no other reason than the process failures.

ISO 17020 should also be rejected on the merits. It will do harm to forensic science practice. Conformity 

assessment inspections are examinations of materials, products, installations, plants, processes, work 

procedures, or services and the determination of their conformity with requirements and the 

subsequent reporting of results of these activities to clients or authorities (ISO 17020/2012). Central to 

this process is the definition of the requirements. These requirements are in essence the conditions 

expected by the client. There are definite presumptions of what should be present for inspection. This is 

antithetical to forensic science work and the scientific method that underlies our work. We approach 

our investigations without preconceived expectations. Equally important, conformity assessments 

simply report deviations from the requirements or expectations. There is no aspect of determination of 

a finding from the data. Central to our work is the application of the scientific method to collect data, 

formulate hypotheses, and test hypotheses. These critical forensic science activities have no role in ISO 

17020 or conformity assessment in general.

The use of a non-forensic, general purpose conformity assessment standard has other unintended and 

negative outcomes. While the accreditation is typically identified at a 17020 accreditation, the 

substance of the forensic science requirements must arise in reference documents identified for a 

particular accreditation. Each and every forensic science body accreditation will be a custom job. This 

means that the courts will need to examine the detailed specifics of an accreditation in each and every 

instance. This puts an unwanted and unnecessarily burdensome requirement on the courts. The custom 

nature of the accreditation can actually result in accreditations with inappropriate scopes that do not 

support quality, complete forensic science work.

Subsequent to the virtual subcommittee’s work, I encountered what I believe to be the first 

accreditation of a fire investigation unit in the US. The accreditation of a fire investigation unit was 

undertaken using 17020 by A2LA. I discussed the process with A2LA, reviewed documentation of their 

process provided to me by A2LA, and conducted a site visit at A2LA to further explore the accreditation 

process and the particular implementation of that process. This process reinforced my concerns with the 

17020 approach to accreditation. I want to thank A2LA for taking the time to educate me concerning 

their processes and their openness to hearing my concerns. Again, I have no reason to question the 

good intentions of A2LA. They were simply doing their best to implement a model approach to forensic 

science unit accreditation defined by their industry group (ILAC).

The accreditation documentation for the subject accreditation states that the accreditation is for 

forensic inspection. It limits the inspection to fire scenes (underline included in the A2LA document). The 

document further defines the inspection as for fire cause and origin determination. It cites NFPA 921 
and NFPA 1033 as well as several ASTM general forensic standards as the Inspection 

Practice/Guide/Procedure definition documents. It cites no particular editions of any of the documents. 

NFPA 921 is the standard of care document within the field of fire and explosion investigation. Similarly, 

NFPA 1033 defines the qualifications and job performance requirements for fire and explosion 

investigators. According to A2LA, the definition of the scope of the accreditation to only fire scene 

inspection was set by the organization seeking accreditation. Because 921’s scope is the entirety of the 

fire and explosion investigation, the accreditation process utilized only the portions of 921 relevant to 

fire scene inspection. The accreditation documentation does not reflect what portions of 921 were used 

in the accreditation. Verbally, A2LA acknowledged that this is problematic. More fundamental, however, 

is that it is professionally inappropriate to reduce the scope of activities to a subset of 921. The fire 

scene examination is merely a subset of the data collection process. There are additional sources of data 

a fire and explosion investigator needs to pursue that are not scene related. Further, the entirety of the 

scientific method process used in 921 goes well outside the fire scene context. Based on 921, a fire 

origin and cause investigation that is limited to fire scene inspection is inherently defective. 

Nonetheless, this is the scope of the accreditation. The accreditation fails to provide clients and courts 

the assurance that the fire investigation unit will provide the services that they require.

Because of the limitations and deficiencies inherent in using 17020 for forensic science, A2LA developed 

specific requirements for forensic examinations (A2LA 318/2015) and an associated checklist (A2LA 

C318/2015). Neither of these documents references any consensus standards for forensic science 

practice. The only referenced document that is not internal to A2LA is ILAC G19:08/2014. In essence 

A2LA needed to develop accreditation requirements for forensic examinations for themselves without 

the benefit of any consensus process. Without judgement with respect to the adequacy of the 

developed requirements, I would suggest that it is inappropriate for accreditation bodies to develop 

themselves the requirements for accreditation. They should limit their work to defining how the 

external consensus document requirements should be implemented in their accreditation processes. To 

A2LA’s credit, they are working to remedy these problems that arise out of both their inexperience and 

the lack of fit by 17020 to forensic investigation.

There seem to be lessons from our experience thus far.

1) First and foremost is that forensic science is not conformity assessment and 17020 is a bad fit with 

forensic science. That is not to say that there may not be elements of 17020 that would serve a forensic 

science standard. These can be included specifically by reference to the relevant section of 17020 or 

included directly within forensic science standards. ISO 17025 should be examined further to evaluate if 

it has similar problems.

2) The forensic science community should define the basis for accreditation through the development of 

consensus standards. Accreditation organizations should simply follow the forensic science community. 

The scope of an accreditation cannot be left to the client to define. They must be forced to select scopes 

that represent complete investigative disciplines that make sense with respect to actual forensic science 

practice and which meet the needs of clients and the courts. Forensic science professional organizations 

must engage with accreditation organizations. Accreditation organizations inherently lack the 

knowledge base to define the accreditation requirements and their implementation. Accreditation 

organizations are process experts, professional organizations are content experts.
3) Accreditation of fire and explosion investigation units is at this time premature. Based upon 

discussions in our SAC, this is likely the case in most if not all crime scene disciplines. In order to 

encourage the development of accreditation processes that are suitable, the OSAC Subcommittee on 

Fire and Explosion Investigation has proposed the development of a new standards making committee 

at NFPA to develop a standard for Fire and Explosion Investigation Units that will be suitable for use by 

accrediting agencies. This consensus standard will be available for use in lieu of 17020 and will 

specifically address the needs of the fire and explosion investigation community and its stakeholders. 

This is the appropriate path for accreditation documents in the forensic investigation community. We 

should avoid the force fit of investigation into a conformity assessment standard.

4) OSAC needs to review its process requirements to assure that candidate documents for registry 

inclusion are robustly discussed and true consensus is obtained. The QIC seems to be an appropriate 

committee to conduct this review. The FSSB needs to take this issue on directly and immediately. The 

reputation of OSAC is at stake and the quality of forensic investigation will suffer if the issue is not 

addressed.

The OSAC consensus process for considering ISO 17020 should be restarted and the whole notion of 

including ISO 17020 in the registry should be reconsidered.