FINAL by: OSAC Program Office; Version 2, January 13, 2020 1

OSAC RESEARCH NEEDS ASSESSMENT FORM

Title of research need:

Detecting Deepfake Multimedia Content

Describe

the need:

Advances in machine learning and computer graphics have made it easier to convincingly 

manipulate video and audio. These so-called Deepfake videos range from complete fullface synthesis and replacement (face- swap), to complete mouth and audio synthesis and 

replacement (lip-sync), and partial word-based audio and mouth synthesis and 

replacement. To get ahead of the problem, there needs to be a way of detecting deep 

fakes, both manually, or through automated means, and span multiple disciplines, 

developing techniques of detection. 

Keyword(s):

Deepfake videos, phoneme-viseme mismatches, full-face synthesis, audio synthesis, mouth 

synthesis, spatial manipulation, temporal manipulation, temporally localized 

manipulations.

Submitting subcommittee(s): VITAL Date Approved: February 26, 2021

(If SAC review identifies additional subcommittees, add them to the box above.)

Background Information:

1. Does this research need address a gap(s) in a current or planned standard? (ex.: Field identification system 

for on scene opioid detection and confirmation)

Not yet but needs to be added.

2. Are you aware of any ongoing research that may address this research need that has not yet been published 

(e.g., research presented in conference proceedings, studies that you or a colleague have participated in but 

have yet to be published)?

S. Agarwal, H. Farid, O. Fried and M. Agrawala, "Detecting Deep-Fake Videos from Phoneme-Viseme Mismatches," 

2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Seattle, WA, USA, 2020, 

pp. 2814-2822, doi: 10.1109/CVPRW50498.2020.00338.

Shruti Agarwal, Hany Farid, Yuming Gu, Mingming He, Koki Nagano, and Hao Li. Protecting world leaders against deep 

fakes. In IEEE Conference on Computer Vision and Pattern Recognition, Workshop on Media Forensics, pages 38–45, 

2019.

M. Turek. Defense Advanced Research Projects Agency. 02/03/2021. https://www.darpa.mil/program/semanticforensics
FINAL by: OSAC Program Office; Version 2, January 13, 2020 2

3. Key bibliographic references relating to this research need: (ex.: Toll, L., Standifer, K. M., Massotte, D., eds. 

(2019). Current Topics in Opioid Research. Lausanne: Frontiers Media SA. doi: 10.3389/978-2-88963-180-3)

Hany Farid. Photo Forensics. MIT Press, 2016.

4. Review the annual operational/research needs published by the National Institute of Justice (NIJ) at 

https://nij.ojp.gov/topics/articles/forensic-science-research-and-development-technology-working-groupoperational#latest? Is your research need identified by NIJ?

No, not found.

5. In what ways would the research results improve current laboratory capabilities?

Some digital/multimedia laboratories have a requirement to conduct authentication examinations of digital & 

multimedia content. Tools which can detect and highlight the presence of a “Deepfake” or other synthetic content 

are needed to support these examinations, especially when new techniques for generating such content are created. 

The existence of Deepfakes will be a necessary concern for examination in the near future as the technology 

advances.

6. In what ways would the research results improve understanding of the scientific basis for the 

subcommittee(s)?

The development of timely Deepfake detection tools and techniques, applied to multimedia content analysis, would 

assist in determining, discrediting, and authenticating the validity of a given multimedia specimen's content.

7. In what ways would the research results improve services to the criminal justice system?

Reliable Deepfake detection could extinguish false attribution with geopolitical, criminal implications, or civil unrest in 

a society whose existence is built upon trust and agreed upon reality.

8. Status assessment (I, II, III, or IV): I

Major gap in 

current 

knowledge

Minor gap in 

current 

knowledge

No or limited

current research is 

being conducted I III

Existing current 

research is being 

conducted II IV

This research need has been identified by one or more subcommittees of OSAC and is being provided as an 

informational resource to the community.