1

OSAC 2023-S-0028 Best Practice Recommendations 

for the Resolution of Conflicts in Toolmark 

Value Determinations and Source Conclusions 

 

OSAC 2023-S-0028 

Best Practice 

Recommendations for the 

Resolution of Conflicts in 

Toolmark Value 

Determinations and Source 

Conclusions 

Firearms & Toolmarks Subcommittee 

Physics/Pattern Scientific Area Committee 

Organization of Scientific Area Committees (OSAC) for Forensic Science 
2

OSAC 2023-S-0028 Best Practice Recommendations 

for the Resolution of Conflicts in Toolmark 

Value Determinations and Source Conclusions 

DRAFT OSAC Proposed Standard 

OSAC 2023-S-0028 

Best Practice Recommendations for 

the Resolution of Conflicts in 

Toolmark Value Determinations 

and Source Conclusions 

Prepared by 

Firearms & Toolmarks Subcommittee 

Organization of Scientific Area Committees (OSAC) for Forensic Science 

Version: 1.0 

September 2023 

Disclaimer

This OSAC Proposed Standard was written by the Organization of Scientific Area Committees 

(OSAC) for Forensic Science following a process that includes an open comment period. This 

Proposed Standard will be submitted to a standards developing organization and is subject to change. 

There may be references in an OSAC Proposed Standard to other publications under development by 

OSAC. The information in the Proposed Standard, and underlying concepts and methodologies, may 

be used by the forensic-science community before the completion of such companion publications. 

Any identification of commercial equipment, instruments, or materials in the Proposed Standard is not a 

recommendation or endorsement by the U.S. Government and does not imply that the equipment, 

instruments, or materials are necessarily the best available for the purpose.

To be placed on the OSAC Registry, certain types of standards first must be reviewed by a Scientific 

and Technical Review (STR). The STR process is vital to OSAC’s mission of generating and 

recognizing scientifically sound standards for producing and interpreting forensic science results. The 

STR shall provide critical and knowledgeable reviews of draft standards or of proposed revisions of 

standards previously published by standards developing organizations (SDOs) to ensure that the 

published methods that practitioners employ are scientifically valid, and the resulting claims are 

trustworthy. 
3

OSAC 2023-S-0028 Best Practice Recommendations 

for the Resolution of Conflicts in Toolmark 

Value Determinations and Source Conclusions 

The STR panel will consist of an independent and diverse panel, including subject matter experts, 

human factors scientists, quality assurance personnel, and legal experts, which will be tasked with 

evaluating the proposed standard based on a comprehensive list of science-based criteria. 

For more information about this important process, please visit our website 

at: https://www.nist.gov/topics/organization-scientific-area-committees-forensic-science/scientifictechnical-review-panels 
4

OSAC 2023-S-0028 Best Practice Recommendations 

for the Resolution of Conflicts in Toolmark 

Value Determinations and Source Conclusions 

Foreword 

This document has been developed with the objective of providing best practice recommendations 

for resolving conflicts with toolmark value determinations or source conclusions that arise during the 

Forensic Science Service Provider’s (FSSP’s) quality assurance process. The potential for differing 

value determinations or source conclusions is possible given the subjective nature of toolmark 

comparisons, particularly for those toolmarks where the quantity and quality of observed data are low 

and require a greater level of interpretation. 

An examiner should be aware of and have access to clear procedures for resolving a conflict. Under 

no circumstances shall any examiner be forced or coerced into agreeing with or writing a technical 

report in support of a source conclusion with which they do not agree. Additionally, FSSP 

management should have processes in place to track the causes and frequency of conflicts between 

examiners. The types, root causes, and frequency of conflicts may illuminate the need for 

supplemental training, additional mentoring, policy and procedure updates, or enhanced monitoring 

of case work. 

The draft of this standard was developed by the Firearms and Toolmarks Subcommittee of the 

Organization of Scientific Area Committees (OSAC) for Forensic Science. 
5

OSAC 2023-S-0028 Best Practice Recommendations 

for the Resolution of Conflicts in Toolmark 

Value Determinations and Source Conclusions 

Table of Contents 

1. Scope Page 6 

2. Normative References Page 6 

3. Terms and Definitions Page 6 

4. General Requirements / Recommendations Page 7 
6

OSAC 2023-S-0028 Best Practice Recommendations 

for the Resolution of Conflicts in Toolmark 

Value Determinations and Source Conclusions 

1 1. Scope 

2 1.1. This document describes the best practice recommendations for resolving conflicts with 

3 toolmark value determinations or source conclusions that may arise between examiners 

4 at any point during the verification process. Throughout this document, the term 

5 “toolmark” is used to refer to both firearm-produced and non-firearm-produced 

6 toolmarks. This document specifies and provides examples for addressing the following: 

7 

8 1.1.1. Conflicting value determinations 

9 

10 1.1.2. Conflicting source conclusions 

11 

12 1.1.3. Documentation and reporting of conflict resolution 

13 

14 1.2. This document does not address differences of opinion that occur during a 

15 consultation or any organizational response once an error is discovered or the 

16 conflict(s) are resolved. 

17 2. Normative References 

18 

19 There are no normative reference documents. 

20 3. Terms and Definitions 

21 For the purposes of this document, the following terms and definitions apply: 

22 

23 3.1. Conflict

24 A condition in which two or more examiners disagree on a value determination or source 

25 conclusion. 

26 

27 3.2. Consultation

28 A discussion between the primary examiner and another qualified examiner for the 

29 purposes of providing guidance/opinion(s)/advice to the primary examiner pertaining to a 

30 source conclusion. This occurs prior to the source conclusion being reached by the 

31 primary examiner. 

32 

33 3.3. Forensic Science Services Provider (FSSP)

34 A forensic science agency or forensic science practitioner providing forensic 

35 science services 

36 

37 3.4. FSSP Management

38 Those designated by the FSSP with the authority and technical expertise to 

39 address and make decisions about conflict resolution matters. Depending on 

40 the FSSP, this may include a technical leader, supervisor, quality manager, etc. 

41 

42 3.5. Observed data

43 Information observed within a toolmark that an examiner relies upon to reach a decision, 

44 conclusion, or opinion. This type of information is commonly expressed as “striae”, 

45 “toolmarks”, “marks of value”, “class characteristics”, or “individual/random 

46 characteristics”, but the use of the broader term “observed data” is inclusive of other types 
7

OSAC 2023-S-0028 Best Practice Recommendations 

for the Resolution of Conflicts in Toolmark 

Value Determinations and Source Conclusions 

47 of data that may be considered, such as reproducibility and range of variability of 

48 toolmark features among a series of marks. 

49 

50 3.6. Primary examiner

51 The firearm and toolmark examiner responsible for conducting a toolmark examination, 

52 reaching source conclusions, and authoring a report. 

53 

54 3.7. Source conclusions

55 Findings or statements expressed as opinion(s) and made by an examiner after 

56 interpretation of observed data. Refer to the scale of source conclusions for 

57 toolmark examinations.

58 

59 3.8. Value determination (suitability for comparison decision)

60 A decision made by an examiner in accordance with FSSP policy or procedure as to 

61 whether or not a questioned item has sufficient features for comparison and a source 

62 conclusion can potentially be reached. 

63 

64 3.9. Verifier

65 The firearm and toolmark examiner tasked with reaching independent opinions or source 

66 conclusions regarding evidence previously examined by the primary examiner. 

67 

68 3.10. Verification

69 An independent comparison or analysis of items previously examined by the primary 

70 examiner to provide a quality check of value determinations or source conclusions. 

71 

72 4. General Requirements / Recommendations 

73 

74 4.1. Conflict resolution may be needed when examiners disagree on a value determination or 

75 a source conclusion. It is recognized that cases often include numerous toolmark 

76 comparisons and could involve more than one conflict resolution process.

77 

78 4.2. FSSPs should have policies in place which contain internal notification requirements 

79 (e.g., notification to the supervisor or quality manager) and processes to track the 

80 causes and frequency of conflicts between examiners.

81 

82 4.3. A conflict may be resolved through a deliberation between the conflicting examiners, 

83 a subsequent blind examination by a third examiner, or some other process as 

84 determined by FSSP management. 

85 

86 4.3.1. Deliberation between the primary examiner and verifier: The primary examiner and 

87 the verifier should attempt to resolve the conflicting value determinations or source 

88 conclusions via deliberation. If agreement is achieved, the conflict resolution 

89 process concludes. If agreement is not achieved, the following actions should be 

90 taken: 

91 

92 4.3.1.1. A conflicting value determination should be resolved by treating the item in 
8

OSAC 2023-S-0028 Best Practice Recommendations 

for the Resolution of Conflicts in Toolmark 

Value Determinations and Source Conclusions 

93 question as if it has value and proceeding with comparison. 

94 

95 4.3.1.2. A conflicting source conclusion should be forwarded to FSSP management 

96 to implement the most appropriate and available option(s) to resolve the 

97 conflict. 

98 

99 4.3.2. When the disagreement between two examiners cannot be resolved through 

100 deliberation between the primary examiner and verifier, the conflict should be 

101 elevated to FSSP management. FSSP management should determine the next 

102 course of action to resolve the conflict. Options may include the following: 

103 

104 4.3.2.1. Examination by a Third Examiner: 

105 

106 4.3.2.1.1. A third examiner (who may be a bench-level examiner, technical leader, 

107 technical supervisor/manager, or an examiner external to the FSSP) 

108 should compare the toolmarks in question and document their own 

109 source conclusion. This should be done blindly, i.e., the third examiner 

110 should be shielded from the decisions, conclusions, and documented 

111 data of the other two examiners. The third examiner should be permitted 

112 to discuss the case with the primary examiner and/or verifier, but only 

113 after the conclusions of the third examiner have been documented. 

114 Whenever practicable, neither the primary examiner nor the verifier 

115 should be the person who chooses or assigns the third examiner. 

116 

117 4.3.2.1.2. The three source conclusions (Those of the primary examiner, verifier, 

118 and third examiner) should be evaluated by FSSP management and an 

119 attempt should be made to determine the reason(s) for the conflict. 

120 FSSP management should then determine how to proceed in 

121 consideration of all available information. Possible options include, but 

122 are not limited to, the following:

123 

124 4.3.2.1.2.1. If FSSP management determines that the primary examiner’s 

125 conclusion is appropriate, the primary examiner may retain the 

126 case and author a report. 

127 

128 4.3.2.1.2.2. If FSSP management determines that the verifier’s conclusion is 

129 appropriate, the case may be reassigned to the verifier, who will 

130 then act as the primary examiner and author a report. 

131 

132 4.3.2.2. Formation of a Scientific Review Panel: 

133 

134 4.3.2.2.1. FSSP management shall define the roles, responsibilities, and 

135 procedures to be followed prior to formation of the scientific review 

136 panel. The FSSP shall then select appropriate subject matter experts 

137 (SMEs) to serve on the panel. SMEs from external organizations may 

138 be used. The SMEs should not have knowledge of the previous 

139 conclusions until they have documented their own conclusions, after 

140 which the SMEs should be permitted to discuss the case with the primary 

141 examiner, verifier, third examiner, and/or other SMEs.
9

OSAC 2023-S-0028 Best Practice Recommendations 

for the Resolution of Conflicts in Toolmark 

Value Determinations and Source Conclusions 

142 4.4. Quality Assurance

143 

144 4.4.1. The FSSP shall determine what quality assurance measures should be taken as a 

145 result of the conflicting value determinations or source conclusions (e.g., root 

146 cause analysis, corrective action).

147 

148 4.5. Documentation and Reporting 

149 

150 4.5.1. The level of documentation needed for conflict resolution will vary according to the 

151 nature of the conflict and FSSP policy. For all conflict resolutions, the following 

152 shall be documented in the case record: 

153 

154 4.5.1.1. The process and outcome of the conflict resolution.

155 

156 4.5.1.2. Written notes and/or images of the observed data to support the 

157 resolution, to include the reasons for the final decision(s). 

158 

159 4.5.1.3. Personal identifier(s), role(s), and date(s) of involvement of all parties 

160 (e.g., examiner, quality manager, supervisor, laboratory director) to the 

161 conflict resolution. 

162 

163 4.5.1.4. All documentation generated by any involved party shall be retained. 

164 This includes all value determinations or source conclusions that were 

165 reached prior to verification and ultimately changed. 

166 

167 4.5.2. If a single conclusion cannot be agreed upon by FSSP management at the end 

168 of the conflict resolution process, the FSSP shall report all conclusions with an 

169 explanation that a conflict arose and could not be resolved.

170 

171 4.5.3. FSSPs should consult legal counsel regarding obligations pertaining to the 

172 disclosure of conflict resolution documentation and/or whether the conflict 

173 should be indicated in the report.