Skip to main content    

An official website of the United States government

Here’s how you know

**Official websites use .gov**              A **.gov** website belongs to an official government organization in the United States.            

**Secure .gov websites use HTTPS**              A **lock** (  LockA locked padlock) or **https://** means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites.            

https://www.nist.gov/itl/ssd/software-quality-group/sdhash-datasets

            -  Topics  -  All Topics 
-  Advanced communications 
-  Artificial intelligence 
-  Bioscience 
-  Buildings and construction 
-  Chemistry 
-  Climate 
-  Cybersecurity 
-  Electronics 
-  Energy 
-  Environment 
-  Fire 
-  Forensic science 
-  Health 
-  Information technology 
-  Infrastructure 
-  Manufacturing 
-  Materials 
-  Mathematics and statistics 
-  Metrology 
-  Nanotechnology 
-  Neutron research 
-  Performance excellence 
-  Physics 
-  Public safety 
-  Resilience 
-  Standards 
-  Transportation 
 
            -  Publications 
            -  Labs & Major Programs  -  Laboratories    -  Communications Technology Laboratory 
  -  Engineering Laboratory 
  -  Information Technology Laboratory 
  -  Material Measurement Laboratory 
  -  Physical Measurement Laboratory 
-  User Facilities    -  NIST Center for Neutron Research 
  -  CNST NanoFab 
-  Research Test Beds 
-  Research Projects 
-  Tools & Instruments 
-  Major Programs    -  Baldrige Performance Excellence Program 
  -  CHIPS for America Initiative 
  -  Manufacturing Extension Partnership (MEP) 
  -  Office of Advanced Manufacturing 
  -  Special Programs Office 
  -  Technology Partnerships Office 
 
            -  Services & Resources  -  Standards and Measurements    -  Calibration Services 
  -  Laboratory Accreditation (NVLAP) 
  -  Quality System 
  -  Standard Reference Materials (SRMs) 
  -  Standards.gov 
  -  Time Services 
  -  Office of Weights and Measures 
-  Software 
-  Data    -  Chemistry WebBook 
  -  National Vulnerability Database 
  -  Physical Reference Data 
  -  Standard Reference Data (SRD) 
-  Storefront 
-  License & Patents 
-  Computer Security Resource Center (CSRC) 
-  NIST Research Library 
 
            -  News & Events  -  News 
-  Events 
-  Blogs 
-  Feature Stories 
-  Awards 
-  Video Gallery 
-  Image Gallery 
-  Media Contacts 
 
            -  About NIST  -  About Us    -  Leadership 
  -  Organization Structure 
  -  Budget & Planning 
-  Contact Us 
-  Visit 
-  Careers    -  Student programs 
-  Work with NIST 
-  History    -  NIST Digital Archives 
  -  NIST Museum 
  -  NIST and the Nobel 
-  Educational Resources 
 

Information Technology LaboratorySoftware and Systems Division

## Software Quality Group

# sdhash Datasets

## Share

Facebook

Linkedin

Twitter

Email

NSRL sdhash 3.3 Data

NSRL has run sdhash version 3.3 (aka "similarity digest") against our unique file corpus. Default values were used in the case of small files which were skipped, and large files which were processed in block mode.

Sdbf files were created for every 1,000 files in each corpus subdirectory.500 sdbf files each were added to individual zip files.40 zip files are available for download, each one representing a span of 500,000 files in the unique file corpus.(Fewer than 500,000 will be found in each zip'ped collection, due to exclusion of small sized files.)

The file names in the sdbf data can be cross-referenced with the corpus metadata.

Here is some example data.

The MD5, SHA1, SHA256 file signatures for the downloads are available here.

Downloads:
```

2.3G   NSRL_corp_000_0.zip 
3.3G   NSRL_corp_000_5.zip
2.9G   NSRL_corp_001_0.zip
1.9G   NSRL_corp_001_5.zip
1.7G   NSRL_corp_002_0.zip
2.0G   NSRL_corp_002_5.zip
4.5G   NSRL_corp_003_0.zip 
4.7G   NSRL_corp_003_5.zip
2.0G   NSRL_corp_004_0.zip
1.9G   NSRL_corp_004_5.zip
1.5G   NSRL_corp_005_0.zip
1.1G   NSRL_corp_005_5.zip
1.9G   NSRL_corp_006_0.zip
2.1G   NSRL_corp_006_5.zip
2.4G   NSRL_corp_007_0.zip
1.9G   NSRL_corp_007_5.zip
1.5G   NSRL_corp_008_0.zip
1.2G   NSRL_corp_008_5.zip
1.9G   NSRL_corp_009_0.zip
1.7G   NSRL_corp_009_5.zip
4.8G   NSRL_corp_010_0.zip
1.3G   NSRL_corp_010_5.zip
2.2G   NSRL_corp_011_0.zip
4.6G   NSRL_corp_011_5.zip
2.8G   NSRL_corp_012_0.zip
1.3G   NSRL_corp_012_5.zip
2.3G   NSRL_corp_013_0.zip
1.6G   NSRL_corp_013_5.zip
1.3G   NSRL_corp_014_0.zip
2.9G   NSRL_corp_014_5.zip
1.5G   NSRL_corp_015_0.zip
1.4G   NSRL_corp_015_5.zip
3.7G   NSRL_corp_016_0.zip
2.3G   NSRL_corp_016_5.zip
2.4G   NSRL_corp_017_0.zip
2.8G   NSRL_corp_017_5.zip
314M   NSRL_corp_018_0.zip
 36M   NSRL_corp_018_5.zip
1.5G   NSRL_corp_019_0.zip
1.8G   NSRL_corp_019_5.zip
0.6G   NSRL_corp_020_0.zip
1.1G   NSRL_corp_020_5.zip
0.5G   NSRL_corp_021_0.zip
1.1G   NSRL_corp_021_5.zip
0.8G   NSRL_corp_022_0.zip
1.4G   NSRL_corp_022_5.zip

```

Digital evidenceSoftware research

## Contacts

                -  Douglas R. White  douglas.white@nist.gov      (301) 975-4761   

### HEADQUARTERS

301-975-2000

Webmaster | Contact Us | Our Other Offices

TwitterFacebookLinkedInInstagramYouTubeGiphyRSS FeedMailing List

Feedback

            -  Site Privacy 
            -  Accessibility 
            -  Privacy Program 
            -  Copyrights 
            -  Vulnerability Disclosure 
            -  No Fear Act Policy 
            -  FOIA 
            -  Environmental Policy 
            -  Scientific Integrity 
            -  Information Quality Standards 
            -  Commerce.gov 
            -  Science.gov 
            -  USA.gov 
            -  Vote.gov