FINAL by: OSAC Program Office; Version 2, January 13, 2020 1

OSAC RESEARCH NEEDS ASSESSMENT FORM

Title of research need: Vehicle Year, Make and Model Analysis Black Box and White Box Studies

Describe

the need:

Forensic image analysts are often asked to determine the make, model and year of a vehicle 

depicted in a questioned image (e.g., CCTV recording) to help advance an investigation. This 

process can also allow an analyst to exclude a suspect vehicle from consideration. Acceptance 

of this type of examination in court can be supported by black box testing to measure the 

accuracy of analysts in this task, particularly at different levels of image quality. The 

inclusion/addition of white box testing can help identify specific features used by analysts in 

reaching their decisions under both “ideal” and less than ideal conditions. The effect of image 

quality and the feature sets determined through these black & white box tests can then be 

incorporated into standards of practice for these exams.

Keyword(s): Vehicle, Make, Model, Year, Automotive, Truck, Car, Motorcycle

Submitting subcommittee(s): VITAL Date Approved: February 26, 2021

(If SAC review identifies additional subcommittees, add them to the box above.)

Background Information:

1. Does this research need address a gap(s) in a current or planned standard? (ex.: Field identification system 

for on scene opioid detection and confirmation)

To date, research in the accuracy of humans in photographic comparison has focused on facial identification. 

However, image analysts are often requested to determine the Make, Model and Year of a vehicle depicted in 

surveillance video or captured images. The VITAL subcommittee seeks to have multiple black box studies conducted 

to determine the accuracy of examiners in making these types of determinations at varying degrees of “Quality” 

within submitted imagery. The standards of practice and strength of opinions (“conclusions”) developed by VITAL can 

be scoped by the results of such black box testing (e.g., strength of opinion may be moderated by lower quality 

images). Likewise, multiple white box tests should also be performed at varying degrees of quality to help define the 

specific features used by analysts to reach their decisions under different quality conditions. Identification of “most” 

and “least” useful features can be incorporated into the standard of practice for conducting a make/model/year 

determination. 

2. Are you aware of any ongoing research that may address this research need that has not yet been published 

(e.g., research presented in conference proceedings, studies that you or a colleague have participated in but 

have yet to be published)?

There have been multiple “case studies” offered over time regarding individual vehicle make/model determinations 

and comparisons, but at this time, no “ground-truthed” research on the accuracy of vehicle make/model/year 

determinations is known. Likewise, there is no known forensic research on defining common characteristics and 

classifiers for make/model/year determination. The Society of Automotive Engineers (SAE) or the self-driving vehicle 

community might have classifiers that can be used. 
FINAL by: OSAC Program Office; Version 2, January 13, 2020 2

3. Key bibliographic references relating to this research need: (ex.: Toll, L., Standifer, K. M., Massotte, D., eds. 

(2019). Current Topics in Opioid Research. Lausanne: Frontiers Media SA. doi: 10.3389/978-2-88963-180-3)

SWGDE Vehicle Make/Model Comparison Form; DOD uses WEFT to identify an aircraft and military equipment.

 

4. Review the annual operational/research needs published by the National Institute of Justice (NIJ) at 

https://nij.ojp.gov/topics/articles/forensic-science-research-and-development-technology-working-groupoperational#latest? Is your research need identified by NIJ?

No.

5. In what ways would the research results improve current laboratory capabilities?

The research results would explore the scientific basis for photographic vehicle comparisons and make/model/year 

determinations, including exploring error rates related to the quality of images and the feature sets found to be most 

diagnostic in the process. For the purposes of these studies, quality could be defined by the percentage of the 

imagery that captures the vehicle or the number of pixels that capture the vehicle, and the number of perspectives of 

the vehicle (multi-camera). Other factors like overall illumination of the scene, contrast, and color fidelity could 

conceivably be incorporated.

6. In what ways would the research results improve understanding of the scientific basis for the subcommittee(s)?

The research would assist in determining which features of a vehicle are most conducive to making a vehicle 

make/model/year determination, as well as assisting in defining the “Quality” of imagery needed for an examiner to 

make an accurate determination of a vehicle make, model and year.

7. In what ways would the research results improve services to the criminal justice system?

The research would help identify ways to improve the ability of examiners to perform make/model determinations for 

lead generation.

8. Status assessment (I, II, III, or IV): I

Major gap in 

current 

knowledge

Minor gap in 

current 

knowledge

No or limited

current research is 

being conducted I III

Existing current 

research is being 

conducted II IV

This research need has been identified by one or more subcommittees of OSAC and is being provided as an 

informational resource to the community.