U.S. flag
        An official website of the United States government
          Here’s how you know
        Here’s how you know**Official websites use .gov**
              
              A **.gov** website belongs to an official government organization in the United States.
        **Secure .gov websites use HTTPS**
              
              A **lock** (  A locked padlock
) or **https://** means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites. https://www.nist.gov/itl/itl-speakers-bureau-peter-bajcsy
  Search NIST
    Search
              Menu

      


Close
    Topics
        All Topics
                                                        Advanced communications
                                                        Artificial intelligence
                                                        Bioscience
                                                        Buildings and construction
                                                        Chemistry
                                                        Climate
                                                        Cybersecurity
                                                        Electronics
                                                        EnergyEnvironment
                                                        Fire
                                                        Forensic science
                                                        Health
                                                        Information technology
                                                        Infrastructure
                                                        Manufacturing
                                                        Materials
                                                        Mathematics and statisticsMetrology
                                                        Nanotechnology
                                                        Neutron research
                                                        Performance excellence
                                                        Physics
                                                        Public safety
                                                        Resilience
                                                        Standards
                                                        Transportation
            Publications
            Labs & Major Programs
        Laboratories
                                Communications Technology Laboratory
                                  Engineering Laboratory
                                  Information Technology Laboratory
                                  Material Measurement Laboratory
                                  Physical Measurement LaboratoryUser Facilities
                                NIST Center for Neutron Research
                                  CNST NanoFab
                                                        Research Test Beds
                                                        Research Projects
                                                        Tools & InstrumentsMajor Programs
                                Baldrige Performance Excellence Program
                                  CHIPS for America Initiative
                                  Manufacturing Extension Partnership (MEP)
                                  Office of Advanced Manufacturing
                                  Special Programs Office
                                  Technology Partnerships Office
            Services & Resources
        Standards and Measurements
                                Calibration Services
                                  Laboratory Accreditation (NVLAP)
                                  Quality System
                                  Standard Reference Materials (SRMs)
                                  Standards.gov
                                  Time Services
                                  Office of Weights and MeasuresSoftware
                                                        Data
                                Chemistry WebBook
                                  National Vulnerability Database
                                  Physical Reference Data
                                  Standard Reference Data (SRD)
                                                        Storefront
                                                        License & PatentsComputer Security Resource Center (CSRC)
                                                        NIST Research Library
            News & Events
        News
                                                        Events
                                                        Blogs
                                                        Feature Stories
                                                        AwardsVideo Gallery
                                                        Image Gallery
                                                        Media Contacts
            About NIST
        About Us
                                Leadership
                                  Organization Structure
                                  Budget & Planning
                                                        Contact Us
                                                        Visit
                                                        Careers
                                Student programsWork with NIST
                                                        History
                                NIST Digital Archives
                                  NIST Museum
                                  NIST and the Nobel
                                                        Educational Resources
 
 Information Technology Laboratory
    
      
  About ITL
                            Expand or Collapse
          ITL Mission
                      ITL Purpose
                      ITL History Timeline
                      Organizational Chart
          How to Work With Us
                            Expand or Collapse
          ITL Opportunities
                      NRC Postdoctoral Research Fellowships in the ITL
                      ITL Grants Program
                      NIST SHIP Program
                      ITL SURF Program
                      Pathways Program
                      Educational Outreach
          Publications
                            Expand or Collapse
          Computer Security Resource Center
                      ITL Patent Policy – Inclusion of Patents in ITL Publications
                      Federal Information Processing Standards (FIPS)
                      NIST Special Publication 800-series General Information
                      NIST Special Publication 1800-series General Information
                      ITL Bulletins
                      ITL Newsletters
                      NIST Special Publication 500 Series
                      Searchable NIST Publication Database
          Priority Areas
                            Expand or Collapse
          Artificial Intelligence
                                              Expand or Collapse
                FARSAIT
                      Cybersecurity
                      Internet of Things
                      Future Computing Technologies and Applications
                      Reliable Computing
          Products and Services
                            Expand or Collapse
          Biometric Special Databases and Software
                      Color FERET Database
                      Computer Security Resource Center
                      Cybersecurity Framework
                      EMNIST Dataset
                      Healthcare - Standards & Testing
                      NIST Digital Library of Mathematical Functions
                      NIST/SEMATECH Engineering Statistics Handbook
                      National Initiative for Cybersecurity Education
                      National Software Reference Library's RDS
                      National Vulnerability Database
                      Tools for Searching the Coronavirus Dataset
                      Voting
          Events
          News & Updates
          Taking Measure Blogs on Information Technology
          Cybersecurity Insights Blog
          Standards Activities
          Videos



  Connect with us
      
      GovDeliveryTwitterITL Speakers Bureau: Peter Bajcsy

Share
  Facebook
                      Linkedin
                      Twitter
                      Email

Title of Talk: MEASUREMENTS OF CLASS ENCODINGS IN NEURAL NETWORKS

ABSTRACT:

This talk will address the problem of measuring class encodings in neural networks (NNs). The work is motivated by understanding adversarial attacks in NNs designed for classification tasks; specifically, NN models trained with poisoned classes embedded into traffic sign images also called trojans. Trojans are defined as physically realizable triggers in input images; for example, a sticky note on top of a STOP sign that causes misclassification of one type of traffic sign into another one, unintended by the NN model design and undesirable by the NN application (e.g., classification of scene objects by self-driving cars).

Our approach is based on (a) designing a web based interactive NN simulator of trojans for visual understanding of two class encodings in input dot patterns [1], (b) extending the measurements of class encodings from simulations to thousand times larger NN models, tens to hundreds of classes, and complex input images generated for four rounds of the TrojAI challenge [2] (4000+ NN models, 22 architectures, 2500 training images per traffic sign class), (c) using the class encodings to generate traceable fingerprints of NN models to training images and to guide designs of trojan detectors [3]. The expected impacts of the work are in the use of (a) publicly accessible NN simulator at https://pages.nist.gov/nn-calculator for educational purposes, (b) class encodings as NN fingerprints for enabling traceability of trained models to training data, and (c) clean and poisoned class encodings for improved design of trojans and trojan detectors.

References:

[1] Peter Bajcsy, Nicholas J. Schaub, and Michael Majurski, “Designing Trojan Detectors in Neural Networks Using Interactive Simulations,” Special Issue on Machine Learning for Cybersecurity Threats, Challenges, and Opportunities, Computing and Artificial Intelligence Section, Appl. Sci. 2021, 11, 0. https://doi.org/10.3390/app11010000; URL: https://www.mdpi.com/journal/applsci/special_issues/cybersecurity_threats

[2] TrojAI Challenge Datasets: https://pages.nist.gov/trojai/docs/about.html

[3] Peter Bajcsy and Michael Majurski, “Baseline Pruning-Based Approach to Trojan Detection in Neural Networks, “ Security and Safety in Machine Learning Systems Workshop at ICLR 2021, URL: https://aisecure-workshop.github.io/aml-iclr2021/cfp (oral presentation), May 7th, 2021

Keywords: neural networks, artificial intelligence-based modeling, computer vision, large scale image processing

BIO:

      Peter Bajcsy Speakers BureauPeter Bajcsy received his Ph.D. in Electrical and Computer Engineering in 1997 from the University of Illinois at Urbana-Champaign (UIUC) and an M.S. in Electrical and Computer Engineering in 1994 from the University of Pennsylvania (UPENN).  He worked for machine vision, government contracting, and research and educational institutions before joining the National Institute of Standards and Technology (NIST) in 2011. At NIST, he has been leading a project focusing on the application of computational science in biological metrology, and specifically stem cell characterization at very large scales. Peter’s area of research is large-scale image-based analyses and syntheses using analytical, statistical, machine learning, and computational models while leveraging computer science foundations for image processing, computer vision, pattern recognition and artificial intelligence. He has authored more than 45 papers in peer reviewed journals and co-authored eight books or book chapters and 110+ conference papers.
  Information technology and Data and informatics

    
    Created March 7, 2022, Updated June 12, 2023
 HEADQUARTERS
            100 Bureau Drive
              Gaithersburg, MD 20899301-975-2000
            Webmaster | Contact Us | Our Other Offices
        Twitter
            
Facebook
            
LinkedIn
            
Instagram
            
YouTube
            
Giphy
            
RSS Feed
            
Mailing List
                              How are we doing? Feedback

  
  Site Privacy
                      Accessibility
                      Privacy Program
                      Copyrights
                      Vulnerability Disclosure
                      No Fear Act Policy
                      FOIA
                      Environmental Policy
                      Scientific Integrity
                      Information Quality Standards
                      Commerce.gov
                      Science.gov
                      USA.gov
                      Vote.gov