FINAL by: OSAC Program Office; Version 2, January 13, 2020 1

OSAC RESEARCH NEEDS ASSESSMENT FORM

Title of research need: Vehicle Comparison Black Box and White Box Studies 

Describe

the need:

Forensic image analysts are often asked to determine if a vehicle depicted in a questioned 

image (e.g., CCTV recording) can be identified or excluded as being a suspect vehicle. 

Acceptance of this type of examination in court can be supported by black box testing to 

measure the accuracy of analysts in this task, particularly at different levels of image 

quality. The inclusion/addition of white box testing can help identify specific features used 

by analysts in reaching their decisions under both “ideal” and less than ideal conditions. 

The effect of image quality and the feature sets determined through these black & white 

box tests can then be incorporated into standards of practice for these exams. 

Keyword(s): Photographic Comparison, Vehicle, patterned evidence, image analysis

Submitting subcommittee(s): VITAL Date Approved: February 26, 2021

(If SAC review identifies additional subcommittees, add them to the box above.)

Background Information:

1. Does this research need address a gap(s) in a current or planned standard? (ex.: Field identification system 

for on scene opioid detection and confirmation)

To date, research in the accuracy of humans in photographic comparison has focused on facial identification. 

However, image analysts are often requested to compare vehicles to determine whether a vehicle depicted in 

surveillance video or captured images is the same as a vehicle linked to a suspect. The VITAL subcommittee seeks to 

have multiple black box studies conducted to determine the accuracy of examiners in making these types of 

comparisons at varying degrees of “Quality” within submitted imagery. The standards of practice and strength of 

opinions (“conclusions”) developed by VITAL can be scoped by the results of such black box testing (e.g., strength of 

opinion may be moderated by lower quality images). Likewise, multiple white box tests should also be performed at 

varying degrees of quality to help define the specific features used by analysts to reach their decisions under different 

quality conditions. Identification of “most” and “least” useful features can be incorporated into the standard of 

practice for conducting a comparison examination.

2. Are you aware of any ongoing research that may address this research need that has not yet been published 

(e.g., research presented in conference proceedings, studies that you or a colleague have participated in but 

have yet to be published)?

There have been multiple “case studies” offered over time regarding individual comparisons, but at this time, no 

known ground-truthed research on the accuracy of vehicle comparisons exists. 

3. Key bibliographic references relating to this research need: (ex.: Toll, L., Standifer, K. M., Massotte, D., eds. 

(2019). Current Topics in Opioid Research. Lausanne: Frontiers Media SA. doi: 10.3389/978-2-88963-180-3)

SWGDE Technical Overview for Forensic Image Comparison; SWGDE Vehicle Make/Model Comparison Form

 
FINAL by: OSAC Program Office; Version 2, January 13, 2020 2

4. Review the annual operational/research needs published by the National Institute of Justice (NIJ) at 

https://nij.ojp.gov/topics/articles/forensic-science-research-and-development-technology-working-groupoperational#latest? Is your research need identified by NIJ?

No.

5. In what ways would the research results improve current laboratory capabilities?

The research results would explore the scientific basis for photographic vehicle comparisons, including exploring error 

rates related to the quality of images and the feature sets found to be most diagnostic in the comparison process. For 

the purposes of these studies, quality could be defined by the percentage of the imagery that captures the vehicle or 

the number of pixels that capture the vehicle, and the number of perspectives of the vehicle (multi-camera). Other 

factors like overall illumination of the scene, contrast, and color fidelity could conceivably be incorporated.

6. In what ways would the research results improve understanding of the scientific basis for the 

subcommittee(s)?

The research would assist in determining which features of a vehicle are most conducive to making a vehicle 

comparison, as well as assisting in the determination of what “Quality” of imagery is needed for an examiner to make 

an accurate determination of a vehicle make, model and year. In addition, the study would assist in the determination 

of error rates in vehicle comparisons.

7. In what ways would the research results improve services to the criminal justice system?

The research would help determine the probative value of vehicle comparison analysis.

8. Status assessment (I, II, III, or IV): I

Major gap in 

current 

knowledge

Minor gap in 

current 

knowledge

No or limited

current research is 

being conducted I III

Existing current 

research is being 

conducted II IV

This research need has been identified by one or more subcommittees of OSAC and is being provided as an 

informational resource to the community.