hemu1438/forensicsgpt-training-data huggingface dataset:
SHA2-256 and SHA3-256 – Digital Corpora
The hash values provided on downloads listing pages are hashes of the file contents, and can be used to verify you’ve downloaded matches the file we hashed. The algorithms we use are:
SHA2-256 as defined in FIPS 180 SHA3-256 as defined in FIPS 202 
We also provide:
Size The size of the file in bytes. AWS S3 ETag The ETag for the https download. AWS uses ETags that are derrived from the MD5 hash algorithm, but they are clearly not MD5 hash codes. Please note that the double-quote (“) is defined by the RFC as being part of the ETag. 
Our database stores (s3key,size,mtime,ETag,sha2_256,sha3_256) as a tuple. When listing S3 keys (files) underneath a given prefix, if the s3 object that is listed matches on size, mtime and ETag, then the sha2_256 and sha3_256 hash codes are displayed. Otherwise they are not.
We periodically dump the database and the S3 bucket metadata underneath the hashes/ prefix.
If you are downloading a file that is a disk image, such as test_image.E01, then the hash value provided on the downloads listing is the cryptographic hash of the E01 file itself, and not of the contained disk image.
S3 Information – Digital Corpora
The Digital Corpora project gets free hosting for the corpus as part of the AWS Open Data Sponsorship Program, for which we are grateful! We could not make this resource available without Amazon’s help.
Accessing the Corpus!
All of our site data is stored in the Amazon S3 bucket s3://digitalcorpora/.
You can download directly from that bucket. We recommend using the bucket directly in Amazon’s cloud, which will give you the fastest access to the data.
The Fast JavaScript Gateway
You can browse the S3 bucket directly using our JavaScript-based browser here: [S3 Browser]. It is fast, but it will not work with wget -r to download many files at once.
The Annotated S3 Gateway
You can also browse using our server-based S3 gateway: [S3 Gateway]. It’s written in python and hosted on Dreamhost, the same ISP that hosts this WordPress site. It provides additional annotations, such as cryptographic hash codes and viewing of the README files. You can find the source code here. If there is a problem with the downloads site, you can try the development site or the backup site
The Unix command line
You can also access this resource from the AWS command line interface with the command:
$ aws s3 ls s3://digitalcorpora/corpora/
                           PRE bin/
                           PRE drives/
                           PRE drives_bulk_extractor/
                           PRE drives_dfxml/
                           PRE files/
                           PRE hashes/
                           PRE mobile/
                           PRE packets/
                           PRE ram/
                           PRE scenarios/
                           PRE sql/
2020-11-21 10:56:19         43 README.txt
2020-11-21 10:56:20    1783404 digitalcorpora.org-hashdeep-2020-04-01.csv
2020-11-21 10:56:19    1787101 digitalcorpora.org-hashdeep-2020-05-01.csv
2020-11-21 10:56:19    1794086 digitalcorpora.org-hashdeep-2020-06-01.csv
2020-11-21 10:56:19    1794914 digitalcorpora.org-hashdeep-2020-07-01.csv
2020-11-21 10:56:20    1796103 digitalcorpora.org-hashdeep-2020-08-01.csv
2020-11-21 10:56:20    1796275 digitalcorpora.org-hashdeep-2020-09-01.csv
2020-11-21 10:56:20    1796447 digitalcorpora.org-hashdeep-2020-10-01.csv
2020-11-21 10:56:20    1796619 digitalcorpora.org-hashdeep-2020-11-01.csv
$
Download Statistics
You can find download statistics at https://stats.digitalcorpora.org/reports.
Tech Details
https://downloads.digitalcorpora.org/ is an app run on a Dreamhost virtual domain that has a Python program which makes the contents of the Amazon S3 bucket s3://digitalcorpora/ look like a Unix directory that is delivered using the Apache web server’s directory listing facility.
The s3://digitalcorpora/ website is hosted in the AWS region US West (Oregon) us-west-2.
The fastest way to access the corpus is by creating an EC2 VM in us-west-2 and accessing it directly.
About DigitalCorpora – Digital Corpora
DigitalCorpora.org is a website of digital corpora for use in computer forensics education research. All of the disk images, memory dumps, and network packet captures available on this website are freely available and may be used without prior authorization or IRB approval. We also have available a research corpus of real data acquired from around the world. Use of that dataset is possible under special arrangement.
The Digital Corpora project gets free hosting for the corpus as part of the AWS Open Data Sponsorship Program, for which we are grateful! We could not make this resource available without Amazon’s help.
Digital Corpora was started by Simson Garfinkel when he was an Associate Professor at the Naval Postgraduate School. The original funding for this project came from what is now called the Software Quality Group at the National Institute of Standards and Technology. The funding created our initial set of corpora and produced the paper, “Bringing science to digital forensics with standardized forensic corpora,” by Simson Garfinkel, Paul Farrell, Vassil Roussev and George Dinolt.
Digital Corpora is a website filled with realistic data for use in digital forensics research and education. Unlike other datasets, all of the information on this website was either created for this purpose or previously made freely available. As such, it can be used in education and research without concern about the existence of sensitive personally identifiable information (PII). No IRB approval is required to work with these data.
In 2023 and 2024, the Digital Corpora website underwent a significant expansion with the donation of the SAFEDOCS and UNSAFE-DOCS corpora that were produced by the DARPA SafeDocs program.
Questions regarding this website should be directed to Simson Garfinkel.
Bibliography – Digital Corpora
Corpora-related publications
Garfinkel, S. Digital Forensics XML and the DFXML toolset, Digital Investigation, 2012.
Woods, Kam, Christoper Lee, Simson Garfinkel, Extending Digital Repository Architectures to Support Disk Image Preservation and Access, JCDL 2011, June 13-17, 2011, Ottawa, Canada.
Woods, K., Christopher Lee, Simson Garfinkel, David Dittrich, Adam Russell, Kris Kearton, Creating Realistic Corpora for Forensic and Security Education, 2011 ADFSL Conference on Digital Forensics, Security and Law
Garfinkel, S., Paul Farrell, Vassil Roussev, George Dinolt, Bringing science to digital forensics with standardized forensic corpora DFRWS 2009
Garfinkel, Simson., Automating Disk Forensic Processing with SleuthKit, XML and Python, Systematic Approaches to Digital Forensics Engineering (IEEE/SADFE 2009), Oakland, California.
Contact – Digital Corpora
Fill in the form below to request the password for the instructor’s packet. If you don’t hear back in a week, please contact Simson Garfinkel directly using the contact information on his personal website.
STUDENTS! Don’t use this form Solutions are only sent to faculty at accredited institutions and government training organizations. Students asking for solutions will be ignored; students impersonating faculty members will be reported to their institutions! (Don’t test us — we’ve done this before.)
Faculty — Please use your official email address at your accredited university or government agency. Forms submitted with GMAIL, YAHOO, HOTMAIL, or other consumer email providers will be ignored.
NOTE: The response will come from the email address corpora-admin@digitalcorpora.org. If you do not get a response within three days after submitting your request, please check your spam box. If that doesn’t work, please resubmit. We have had problems sending email to certain .mil domains.
Frequently asked questions
Q: I am currently studying digital forensics taking different online courses and I am interested if possible to get the passwords for the solutions.
A: Unfortunately, we only provide solutions to faculty at accredited educational institutions and to various government organizations.
Q: I am an instructor, but my name does not appear on my educational institution’s website
A: Please provide the URL of the course that you are teaching or the program that you are in.
Terms of Use – Digital Corpora
Unless otherwise notice, all original content on this site, including metadata, original scenario data, and teachers guides, are made available under the CC0 license. CC0 is in effect because these materials were made by US Government employees as part of their official duties.
Any material that explicitly claims a copyright is not subject to the CC0 license. This includes copyrighted software and other original materials located in disk images, packet dumps, and other digital artifacts located on this website.


To the extent possible under law, Simson L. Garfinkel has waived all copyright and related or neighboring rights to Digital Corpora Website and Materials. This work is published from: United States.
File Corpora – Digital Corpora
Leave a Reply 
Your email address will not be published. Required fields are marked *
Comment * 
You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> 
 <strike> <strong> 
Save my name, email, and website in this browser for the next time I comment.
May 2023 – Digital Corpora
CIRCL Forensics Exercises are little challenges developed for and during the CIRCL Forensics Trainings, and for workshops or presentations. Usually you will find a PDF with the slides and the solution inline, next to a disk image with the challenge itself.
Wiped Disk Image:
https://downloads.digitalcorpora.org/corpora/drives/circl-2023-wiped/
Recovering data from a wiped disk sounds impossible. But wiping a ‘big’ disk would take time. If the adversary is not patient and interrupts the wiping process after some limited time, there is a good chance to recover some plain data, or even complete file system structures and/or partitions. Goal of this exercise is, to recover data from a partially wiped disk image.
This is a manual approach, to learn how to analyze data on byte level.
To be continued
Search Govdocs1 – Digital Corpora
Sadly, the search facility is currently not working.
Leave a Reply 
Your email address will not be published. Required fields are marked *
Comment * 
You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> 
 <strike> <strong> 
Save my name, email, and website in this browser for the next time I comment.
This site uses Akismet to reduce spam. Learn how your comment data is processed.
June 2020 – Digital Corpora
A new set of iOS 13 images and files have been contributed to the collection by Joshua Hickman. Details can be found here (13.3.1) and here (13.4.1)
I am pleased to announce that a new scenario has been added to the digitalcorpora.org database! This scenario was created in collaboration with Paul Bryant of the Wellington Institute of Technology. 
The details of this scenario can be found here.
A new corpus has been added to the collection by Sven Schmitt and Sebastian Nemetz. A link to the details encompassing this corpus can be found here.
November 2020 – Digital Corpora
You can download any file using this new base URL: https://downlods.digitalcorpora.org/corpora/
For example, corpus file drives/nps-2014-xbox1/narrative.txt can be downloaded from:
https://downloads.digitalcorpora.org/corpora/drives/nps-2014-xbox1/narrative.txt
You can browse the DigitalCorpora S3 bucket using our browser at https://downloads.digitalcorpora.org/corpora/.
You can also use our JavaScript browser: https://digitalcorpora.org/s3_browser.html
The DigitalCorpora project recently joined the AWS Open Data Sponsorship Program. This means that big changes are underway, and they are all for the better!
Under the Open Data program, we will be transitioning the digitalcorpora corpus to storage in Amazon S3. We will have a gateway to preserve the existing links at https://downloads.digitalcorpora.org/corpora/, but the gateway will redirect you to an AWS S3 download. We will also be adding TTS for downloads.
This means that the download server will be more stable moving forward. It will also make it much easier (and faster) to analyze the corpus from the Amazon Cloudâ€”and all forensics should be done in the cloud, right?
We’ve been hosting the downloads server at George Mason University since 2015 and it’s been a great ride! We will probably keep the GMU downloads server operational as our backup downloads server for the next year or two.
As preparation for the migration, we will be removing all symbolic links from the downloads server. Currently, the following symbolic links are in use:
dcorpora@digitalcorpora:~/corpora$ ls -l `find . -type l -print`
lrwxrwxrwx 2 dcorpora apache  6 May 16  2015 ./cell-phones -> mobile
lrwxrwxrwx 2 dcorpora apache  6 May 16  2015 ./disk-images -> drives
lrwxrwxrwx 1 dcorpora apache 22 Aug 19  2019 ./mobile/2019-owl -> ../scenarios/2019-owl/
lrwxrwxrwx 2 dcorpora apache  7 May 16  2015 ./network-packet-dumps -> packets
lrwxrwxrwx 1 dcorpora apache 33 Jul 18 09:16 ./scenarios/2008-m57-jean -> ../disk-images/nps-2008-m57-jean/
lrwxrwxrwx 1 dcorpora apache 23 Aug 18  2019 ./scenarios/2008-nitroba -> ../packets/2008-nitroba
lrwxrwxrwx 2 dcorpora apache 42 May 17  2015 ./scenarios/2009-m57-patents/drives-redacted -> ../../drives/nps-2009-m57-patents-redacted
lrwxrwxrwx 2 dcorpora apache 30 May 17  2015 ./scenarios/2009-m57-patents/net -> ../../packets/2009-m57-patents
lrwxrwxrwx 2 dcorpora apache 30 May 17  2015 ./scenarios/2009-m57-patents/ram -> ../../ram/nps-2009-m57-patents
lrwxrwxrwx 2 dcorpora apache 33 May 17  2015 ./scenarios/2009-m57-patents/usb -> ../../drives/nps-2009-patents/usb
int dcorpora@digitalcorpora:~/corpora$
This will be resolved as followings:
The cell-phones, disk-images and network-packet-dumps links will be removed.
The proper home for 2019-owl will be under scenarios/; a file 2019-owl.txt will be left behind in the /mobile directory.
All of the m57 data will be moved under the scenario. As above, text files will be left behind
March 2013 – Digital Corpora
The following post is now obsolete. The file frequent_hashcodes_and_paths_rdc.xml has been removed from the corpus as explained in a more recent post. Please see:
https://digitalcorpora.org/archives/391
Deprecated Post from Mar 29, 2013 @ 13:13
The file frequent_hashcodes_and_paths_rdc.xml contains SHA1 hashcode and path data derived from the Real Drive Corpus collected by the DEEP Project at the U.S. Naval Postgraduate School. The file provides two kinds of data useful to forensic investigators: (1) SHA1 hashcodes that occurred for undeleted files on at least five different drives in the corpus but did not occur in the National Software Reference Library (https://www.nsrl.nist.gov). These are likely to indicate files uninteresting and excludable in most forensic investigations. File sizes and names are also given. (2) Path names (file name plus all directories) for paths that occurred on at least twenty different drives in the corpus on undeleted files. These usefully supplement the hashcodes in indicating recurring files uninteresting for investigators. However, occurrences of these files could include viruses and other malware, or could be hiding illegal content although it is unlikely.
Read more … https://digitalcorpora.org/corp/hashes/nus-deidentified/README-frequent-hashcodes-and-paths-rdc.txtDownload XML File (HAS BEEN REMOVED): https://digitalcorpora.org/corp/hashes/nus-deidentified/frequent-hashcodes-and-paths-rdc.xml (102 MB)
April 2013 – Digital Corpora
File bulk_extractor-1.3.1.zip contains the source code for bulk_extractor v1.3.1.Â bulk_extractor is a C++ program that scans a disk image, a file, or a directory of files and extracts useful information without parsing the file system or file system structures.Â bulk_extractor is typically downloaded on a Fedora system and compiled or cross-compiled to Linux, Mac, or Windows using autotools.Â Please seeÂ https://github.com/simsong/bulk_extractor/wiki/Introducing-bulk_extractor.
BEViewer.jar is an executable bulk_extractor viewer user interface.
Bulk Extractor ViewerÂ (BEViewer) provides a graphical user interface for browsing features that have been extracted via theÂ bulk extractorÂ feature extraction tool.Â Please seeÂ https://github.com/simsong/bulk_extractor/wiki/BEViewer.
be_installer-1.3.exe is a Windows installer for installing bulk_extractor and BEViewer v1.3 on a Windows system.
bulk_extractor.pdf, “Digital media triage with bulk data analysis and bulk-extractor,” discusses how the bulk_extractor tool is effective in providing bulk data analysis.
2012-08-08 bulk_extractor Tutorial.pdfÂ describes how to use the BEViewer tool.Â Although some of the parameters for running bulk_extractor have changed, the majority of the tutorial remains current..
Source: The information above and links were received from Bruce Allen <bdallen@nps.edu>, Naval Postgraduate School
See other bulk_extractor downloads here:Â https://digitalcorpora.org/downloads/bulk_extractor/
October 2013 – Digital Corpora
The text file govdocs1-first512-first4096-docid.txt containing MD5 hashes of the first 512 bytes and first 4096 bytes of every file in the GOVDOCS1 corpus has been removed. This file was provided to assist with research of block hashes. We have since created the hashdb toolset which provides support for creating and working with hash block databases.Â Please refer to https://github.com/simsong/hashdb/wiki for downloading the code, continuing progress on this topic, and links to relevant papers including:
Distinct Sector Hashes for Target File Detection
A related masters thesis on this topic was completed at Naval Postgraduate School in 2012 and can be downloaded for additional reading: https://simson.net/ref/2012/kmf_thesis.pdf
February 2014 – Digital Corpora
UT San Antonio has kindly provided digitalcorpora with open source, publicly releasable samples of 32 file types. These are the samples that were used by Dr. Nicole Beebe to develop the Sceadan File Type Classifier.
Included file types are ASP, AVI, B64, B85, BZ2, CSS, DLL, ELF, EXE, EXT3, FAT, FLV, JAR, JB2, JS, M4A, MOV, MP3, MP4, NTFS, PST, RPM, RTF, Random, SWF, TXT, Tbird, URL, WAV, WMA, XLSX, ZIP. Each file type sample can be downloaded from the website:
* https://downloads.digitalcorpora.org/corpora/files/filetypes1/
Also included is a _README directory that includes a list of every file downloaded and a copyright statement for the files that are covered under copyright. You can access that directory at:
* https://downloads.digitalcorpora.org/corpora/files/filetypes1/_README/
This “FLETYPES1” corpus supplements the files in the GOVDOCS1 corpus.
Please let us know if you use these by including this citation in your paper:
“FILETYPES1 File type samples,” Beebe, Nicole, University of Texas, San Antonio, hosted at https://downloads.digitalcorpora.org/corpora/files/filetypes1/. 2014
February 2011 – Digital Corpora
I have created a new disk image called 2010-nps-emails that can be used for testing programs that find email addresses or perform string search.
The disk image consists of 30 different email addresses, each one stored in a different document with a different coding scheme.
Below are a list of the email addresses and their codings:
email address                             Application (Encoding)

plain_text@textedit.com                   Apple TextEdit  (UTF-8)
plain_text_pdf@textedit.com               Apple TextEdit print-to-PDF (/FlateDecode)
rtf_text@textedit.com                     Apple TextEdit (RTF)
rtf_text_pdf@textedit.com                 Apple TextEdit print-to-PDF (/FlateDecode)
plain_utf16@textedit.com                  Apple TextEdit (UTF-16)
plain_utf16_pdf@textedit.com              Apple TextEdit print-to-PDF (/FlateDecode)

pages@iwork09.com                         Apple Pages '09
pages_comment@iwork09.com                 Apple Pages (comment) '09
keynote@iwork09.com                       Apple Keynote '09
keynote_comment@iwork09.com               Apple Keynote '09 (comment)
numbers@iwork09.com                       Apple Numbers '09
numbers_comment@iwork09.com               Apple Numbers '09 (comment)

user_doc@microsoftword.com                Microsoft Word 2008 (Mac) (.doc file)
user_doc_pdf@microsoftword.com            Microsoft Word 2008 (Mac) print-to-PDF
user_docx@microsoftword.com
user_docx_pdf@microsoftword.com           Microsoft Word 2008 (Mac) print-to-PDF (.docx file)
xls_cell@microsoft_excel.com
xls_comment@microsoft_excel.com           Microsoft Word 2008 (Mac)
xlsx_cell@microsoft_excel.com             Microsoft Word 2008 (Mac)
xlsx_comment@microsoft_excel.com          Microsoft Word 2008 (Mac) (Comment)

doc_within_doc@document.com               Microsoft Word 2007 (OLE .doc file within .doc)
docx_within_docx@document.com             Microsoft Word 2007 (OLE .doc file within .doc)
ppt_within_doc@document.com               Microsoft PowerPoint and Word 2007 (OLE .ppt file within .doc)
pptx_within_docx@document.com             Microsoft PowerPoint and Word 2007 (OLE .pptx file within .docx)
xls_within_doc@document.com               Microsoft Excel and Word 2007 (OLE .xls file within .doc)
xlsx_within_docx@document.com             Microsoft Excel and Word 2007 (OLE .xlsx file within .docx)

email_in_zip@zipfile1.com                 text file within ZIP
email_in_zip_zip@zipfile2.com             ZIP'ed text file, ZIP'ed
email_in_gzip@gzipfile.com                text file within GZIP
email_in_gzip_gzip@gzipfile.com           GZIP'ed text file, GZIP'ed
The image can be downloaded from nps-2010-emails
Edit, 2011-11-26 19:32 PST: One email was incorrectly recorded above. xlsx_comment@microsoft_excel.com is within the disk image, but xlsx_cell_comment@microsoft_excel.com was recorded here. That is now corrected above.
February 2021 – Digital Corpora
We have transitioned https://downloads.digitalcorpora.org/corpora/ from George Mason University to Amazon S3. The transition should be completed by 0445 GMT Feb 3, 2021. When the transition is finished, you will also be able to access digitalcorpora at https://downloads.digitalcorpora.org/corpora/.
The old downloads system can still be accessed at https://gmu.digitalcorpora.org/corpora/
March 2012 – Digital Corpora
We have created a tarÂ and a ZIP file with 109,223 files from the govdocs1m corpus. You can download them from:
https://downloads.digitalcorpora.org/corpora/files/govdocs1/by_type/files.jpeg.tarÂ Â [37.6 GB]
Browse all by type: https://downloads.digitalcorpora.org/corpora/files/govdocs1/by_type/
Please note that the ZIP file is necessarily a ZIP-64 file and will not decompress with the ZIP implementation built-in to MacOS or Windows.