U.S. flag
      An official website of the United States government, Department of Justice.
        Here's how you know
      Here's how you know**Official websites use .gov**
            
            A **.gov** website belongs to an official government organization in the United States.
      **Secure .gov websites use HTTPS**
            
            A **lock** (
            A locked padlock
            ) or **https://** means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites. Menu

      
        close

            
            
                                              About NIJ
        
                    
  
          About NIJ
      
    
          The NIJ Director
          How NIJ Is Organized
          Mission and Guiding Principles
          Research and Development Process
          Strategic Framework
          Assessments of NIJ
          Annual Reports & Reports to Congress
          Staff Directory
          Funding & Awards
        
                    
  
          Funding & Awards
      
    
          Current Funding
          Forthcoming Funding
          Expired Funding
          Awards: Listing of Funded Projects
          Guidance for Applicants and Awardees
          Fellowship & Student Programs
          Travel Scholarships
          NIJ Challenge Program
          Library & Multimedia
        
                    
  
          Library & Multimedia
      
    
          NIJ Journal
          Podcasts
          Publications Listing
          Multimedia Listing
          Rated Practices on CrimeSolutions
          Rated Programs on CrimeSolutions
          Software & Databases
          Datasets
          Patents
          Topics
        
                    
  
          Topics
      
    
          Article Listing
          Corrections
          Courts
          Crime Prevention
          Crimes
          Drugs and Crime
          Equipment and Technology
          Forensic Sciences
          Youth Justice
          Justice System Reform
          Law Enforcement
          Tribal Crime and Justice
          Victims of Crime
          Events
        
                    
  
          Events
      
    
          NIJ 2024 Research Conference
          Scheduled Events
          On-Demand Events
          Research Meetings & Workshops
          NIJ-Hosted Online Training

                            
                                About
          Contact Us
          Subscribe
          Topics A-Z
                  Share
FacebookXLinkedInRedditPinterestEmail Home
                            Funding & Awards
 Audio Deepfake Detection for Forensics and Security
Award Information
Awardee
              UNIVERSITY OF ROCHESTER
Award #
              15PNIJ-23-GG-01933-RESS
Funding Category
              Competitive Discretionary
Location
                    ROCHESTER, NY
Congressional District
              25
Status
              Open
Funding First Awarded
              2023
Total funding (to date)
              $64,003
Original Solicitation
              NIJ FY23 Graduate Research Fellowship
Description of original award (Fiscal Year 2023, $64,003)
The rapid development of AI-powered audio generation has made it increasingly difficult
to distinguish between real and fake audio, leading to the use of deepfakes for criminal
activities. Therefore, there is a pressing need to develop audio deepfake detection
(ADD) systems that can filter out deepfakes. To achieve accurate detection results, ADD
systems need to be robust against new and unknown deepfake techniques, provide
evidence to support their findings, and be compatible with other fake detection tools.
This grant proposal aims to investigate three key research questions (RQs) in ADD for
forensics and security:
1. How can we design ADD systems that are robust to new and unknown deepfake
techniques?
2. How can we improve the convincingness and explainability of ADD results?
3. Can we leverage other modalities, such as vision, to enhance ADD performance?
To address the first RQ, the applicant proposes a novel training strategy called multicenter
one-class learning that considers the distribution mismatch between training and
evaluation. The proposed training strategy encourages the embedding space of speech
representations to separate deepfakes and cluster real recordings of diverse settings
around multiple centroids, leading to ADD systems that can outperform existing models
in detecting novel deepfake techniques.
To address the second RQ, the applicant proposes to identify fingerprints of the audio
generation process and use them as evidence for convincing ADD results. The
applicant proposes a proactive steganography framework that embeds the description
of deepfake generation algorithms into the generated audio, enabling authorized agents
to accurately determine the origin of fraudulent speech, even for novel deepfakes.
To address the third RQ, the applicant proposes to leverage cross-modal learning to
enhance ADD performance with information from other senses such as vision. The
applicant proposes modeling cross-modal mismatches on the synchronization of
general audio-visual events in complex scenes, including talking head and surveillance
videos. Incorporating knowledge graph reasoning on the acoustic properties can further
improve the ADD performance.
The proposed research will contribute to the development of effective ADD systems that
are robust against new and unknown deepfake techniques, provide evidence to support
their findings, and are compatible with other fake detection tools. As deepfakes become
more prevalent in crimes, the efficiency of audio forensics in juvenile justice can be
significantly improved through the development of accurate and robust ADD techniques.
The publications resulting from this research will be made publicly available to
interested juvenile justice practitioners and the broader public. CA/NCF

  
Date Created: September 8, 2023
    
          Side navigation
  

        

  
  
              Current Funding
    
      
            
            
            
      Forthcoming Funding
    
      
            
            
            
      Expired Funding
    
      
            
            
            
      Awards: Listing of Funded Projects
    
      
            
            
            
      Guidance for Applicants and Awardees
    
      
            
            
            
      Fellowship & Student Programs
    
      
            
            
            
      Travel Scholarships
    
      
            
            
            
      NIJ Challenge ProgramSimilar Awards
        
                      Comparison of Examiner Conclusions with Score-based Likelihood Ratios for Cartridge Case Evidence
                  Experimental and numerical investigations for the development and validation of thermo-chemistry models and property databases for selected wall lining materials under fire exposure
                  Developing a Strategy to Evaluate Urban Violence Prevention and Intervention Ecosystems: A Case Study of Boston
 Top

      National Institute of Justice

        
        
                        About NIJ
                    About NIJ
      
    
                  How NIJ Is Organized
      
    
                  NIJ Director
      
    
                  Research Disclaimer
      
    
                  Funding & Awards
                    Funding & Awards Home
      
    
                  Awards Listing
      
    
                  Current Funding
      
    
                  Guidance for Applicants and Awardees
      
    
                  Resources
                    Events
      
    
                  Podcasts
      
    
                  Publications
      
    
                  Topics
      
    
                  Multimedia
      
    
                  Reuse Policy
      
    
                  Contacts
                    Contact NIJ
      
    
                  Staff Directory
      
    
                  Submit a FOIA Request
      
    
                  File a Civil Rights ComplaintFacebook
          
                      X
          
                      YouTube
  

      810 7th St. NW, Washington, D.C. 20531
      
              Secondary Footer link menu
            Accessibility
              Plain Language
              Privacy Policy
              Legal Policies and Disclaimer
              No FEAR Act
              Freedom of Information Act
              USA.gov
              Justice.gov