hemu1438/forensicsgpt-training-data huggingface dataset:
March 2010 – Digital Corpora
We will be making a presentation and handing out DVDs filled with data at the Open Source Forensics Conference, held in conjunction with the Basis Technology Government User’s Conference, June 8-9, 2010, at the Westfield Marriott in Chantilly, VA.
Our friends at anti-forensics.ru have given us seven very small disk images that are designed to demonstrate failings of particular open source Linux distributions.
You can view all of the images at https://digitalcorpora.org/corp/images/aor/. The images you will find there includes:
2009-aor-test_caine15.iso
2009-aor-test_deft5.iso
2009-aor-test_grml200910.iso
2009-aor-test_othernew.iso
2009-aor-test_otherold.iso
2009-aor-test_raptor20091026.iso
2009-aor-test_spada4.iso
These images should be directly copied to a hard drive or a partition. Forensic Linux distributions would use them as root file systems and execute proof-of-concept code during the boot.
Details of why these images are useful can be found on the author’s website, at: Linux_for_computer_forensic_investigators_2.pdf
Ever want to have SQL access to the NIST RDS but didn’t want to spend a month building the MySQL tables? Well, we did too… So we took one of our 8-core, 32GB servers, imported all of the NSRL, and then put a tar file of the tables available for download on this server.
To use these files just download https://downloads.digitalcorpora.org/corpora/files/nist-nsrl-rds226.tar.bz2 and put the files in your MySQL data directory. You’ll be up-and-running in no time.
We are revising the structure and content of this website. Please let us know if you find any problems.
June 2010 – Digital Corpora
Forensic Innovations, Inc., makers of File Investigator TOOLS, has performed an analysis of the 986,278 files in the “1 million file corpus”. (13,722 files in the corpus were removed earlier this year because they were from California State Government web servers that were in the .gov domain and mistakingly collected as part of the original collection effort.)
We would like to thank Forensic Innovations for their work in support of this project. We have made available their summary report and will be making available their file-by-file analysis as soon as we deploy an appropriate database on this website.
Simple Statistical Report of the 1 million file corpus.
NIST – Digital Corpora
Ever want to have SQL access to the NIST RDS but didn’t want to spend a month building the MySQL tables? Well, we did too… So we took one of our 8-core, 32GB servers, imported all of the NSRL, and then put a tar file of the tables available for download on this server.
To use these files just download https://downloads.digitalcorpora.org/corpora/files/nist-nsrl-rds226.tar.bz2 and put the files in your MySQL data directory. You’ll be up-and-running in no time.
January 2011 – Digital Corpora
I have posted a text file containing MD5 hashes for the first 512 bytes and first 4096 bytes of every file in the GOVDOCS1 corpus. This file is intended for research on sector hashing. You can download the file from https://downloads.digitalcorpora.org/corpora/files/govdocs1/govdocs1-first512-first4096-docid.txt
Network Packet Dumps – Digital Corpora
You will find network packet dumps in the following data sets:
M57-Patents Scenario has packet dumps from each day of operations.
Nitroba University Harassment Scenario consists solely of packets captured at a dorm room.
Over 2000 packet dumps from DEFCON 20 Capture The Flag
A 5GB TCP connection, just the thing for testing TCP/IP reassembly tools
If you are looking for network packets, you may find these pages on other websites useful as well:
The MIT Lincoln Laboratory’s DARPA Intrusion Detection Evaluation data set.
2008 Nitroba University Harassment Scenario – Digital Corpora
(Note: Because packet capture files contain timestamps for each packet, this scenario needs to have a date and time when it takes place. This scenario takes place in Summer 2008. The date and time stamps are not relevant in solving the problem set.)
You are a security administrator at the prestigious (and fictional) Nitroba State University.
Nitroba’s IT department received an email from Lily Tuckrige, a teacher in the Chemistry Department. Tuckrige has been receiving harassing emails and she suspects that they are being sent by a student in her class Chemistry 109, which she is teaching this summer. The email was received at Tuckridge’s personal email account, lilytuckrige@yahoo.com. She took a screenshot of the web browser and sent it in.
The system administrator who received the complaint wrote back to Tuckridge that Nitroba needed the full headers of the email message. Tuckridge responded by clicking the “Full message headers” button in Yahoo Mail and sent in another screen shot, this one with mail headers.
The mail header shows that the mail message originated from the IP address 140.247.62.34, which is a Nitroba student dorm room. Three women share the dorm room. Nitroba provides an Ethernet connection in every dorm room but not Wi-Fi access, so one of the women’s friends installed a Wi-Fi router in the room. There is no password on the Wi-Fi.
Because several email messages appear to come from the IP address, Nitroba decides to place a network sniffer on the ethernet port. All of the packets are logged. On Monday 7/21 Tuckridge received another harassing email. But this time instead of receiving it directly, the perpetrator sent it through a web-based service called “willselfdestruct.com.” The website briefly shows the message to Tuckridge, and then the website reports that the “Message Has Been Destroyed.”
You have been given the screen shots, the packets that were collected from the Ethernet tap, and the Chem 109 roster. Your job is to determine if one of the students in the class was responsible for the harassing email and to provide clear, conclusive evidence to support your conclusion.
The teaching materials for this scenario includes:
pcap file – about 60MB
slides introducing the problem [PDF] [PPT] [Keynote]
Password Protected Teacher’s solution (password information)
Expanded Nitroba Harassment Lab Scenario (Encrypted Solutions) (Thanks to Ajoy Ghosh at the UNSW Canberra for this work!)
Hash values for nitroba.pcap:
Algorithm Value 
MD5	9981827f11968773ff815e39f5458ec8	
SHA1	65656392412add15f93f8585197a8998aaeb50a1	
SHA256	2b77a9eaefc1d6af163d1ba793c96dbccacb04e6befdf1a0b01f8c67553ec2fb	
(Note: Because packet capture files contain timestamps for each packet, this scenario needs to have a date and time when it takes place. This scenario takes place in Summer 2008. The date and time stamps are not relevant in solving the problem set.)
Scenarios – Digital Corpora
Scenarios are collections of multiple disk images, memory dumps, network traffic, and/or data from portable devices. Currently, we have just a few scenarios, but we hope to create (or acquire) more.
2019 Tuck – A scenario involving a person who tries to join a terrorist organization (needs to be written up)
2019 Owl – In a jurisdiction where Owls are illegal to trade and buy, two users are discussing the illegal trade of owls.
2019 Narcos – Due to intelligence provided by the Australian government, two passengers were intercepted by Customs upon arriving at Wellington, New Zealand from Brisbane. The Intel provided stated that Jane Esteban and John Fredricksen may be involved in illegal activity.
2018 Lone Wolf Scenario – A scenario involving the seizure of the laptop of a fictional person planning a mass shooting. (*)
2012 National gallery DC – a fictional attack on the National Gallery DC, foiled in 2012.
2009 M57-Patents – A complex scenario involving multiple drives and actors set at a small company over the course of several weeks. (*)
2008 Nitroba University Harassment Scenario – A fun-to-solve network forensics scenario. (*)
2008 M57-Jean – A single disk scenario involving the exfiltration of corporate documents from an executive’s laptop.
You can also directly browse the scenarios on the web server at:
https://downloads.digitalcorpora.org/corpora/scenarios/
Solutions
Solutions are available for the the (*) indicated scenarios. Please note that solutions are only available to faculty at accredited institutions and to trainers within the US Government. Please do not ask us to make an exception in your case! Information on how to receive the solutions is on the solutions page.
Bibliography
Garfinkel, S., “Lessons Learned Writing Computer Forensics Tools and Managing a Large Digital Evidence Corpus”, DFRWS 2012, Aug. 6-8, 2012, Washington, DC.
Garfinkel, Farrell, Roussev and Dinolt, Bringing Science to Digital Forensics with Standardized Forensic Corpora, DFRWS 2009, Montreal, Canada. (slides) BEST PAPER AWARD. (Acceptance rate: 36%, 15/41)
Woods, Kam, Christoper Lee, Simson Garfinkel, Extending Digital Repository Architectures to Support Disk Image Preservation and Access, JCDL 2011, June 13-17, 2011, Ottawa, Canada. (Acceptance rate: 28%, 28/99 )
Woods, K., Christopher Lee, Simson Garfinkel, David Dittrich, Adam Russel, Kris Kearton, Creating Realistic Corpora for Forensic and Security Education, 2011 ADFSL Conference on Digital Forensics, Security and Law (Acceptance rate: 50%, 32/16)
Packet Dumps – Digital Corpora
Packet dumps on this server:
Nitroba University Harassment Scenario
Daily packet dumps part of the M57 Patents Scenario
DEFCON 19 (2011) and 20 (2012) CTF packet dumps
Packet dumps from other organizations:
DARPA 1998 Intrusion Detection Evaluation Data Set [homepage]
DARPA 1999 Intrusion Detection Evaluation Data Set [homepage]
DARPA 2000 Intrusion Detection Scenario Specific Data Sets [homepage]
2018 Lone Wolf Scenario – Digital Corpora
The 2018 Lone Wolf scenario is a set of materials from the a fictional seizure of a laptop of a fictional individual who was planning a mass shooting. In the scenario, the individual’s brother alerted the police regarding the increasingly concerning behavior of his brother. As a result of the alert, the police seized the brother’s laptop. The laptop was then imaged with the FTK Imager program.
Forensic Data
This scenario was created by Thomas Moore, a student at George Mason University, as his final project for CRFS 780: Cloud Forensics, taught in Spring 2018 by Simson Garfinkel. The purpose of the scenario is to give students the chance to work with a dataset that contains cloud artifacts left on clients, and to provide a scenario with a realistic size.
The seizure consists of the following materials:
FTK Imager Log.txt
LoneWolf.E01 (1.5GB)
LoneWolf.E02 (1.5GB)
LoneWolf.E03 (1.5GB)
LoneWolf.E04 (1.5GB)
LoneWolf.E05 (1.5GB)
LoneWolf.E06 (1.5GB)
LoneWolf.E07 (1.5GB)
LoneWolf.E08 (1.5GB)
LoneWolf.E09 (0.9GB)
memdump.mem (17GB)
pagefile.sys (2.9GB)
Alternatively, you can download these materials in two different large ZIP files:
single 15GB ZIP file containing the disk images.
single 32GB ZIP file containing the disk and memory images.
Forensic Reports
Because some students do not have access to commercial forensic tools, this scenario comes complete with the reports from several such tools. You can find the list of all tool reports here.
Lone Wolf Forensic Outputs
Solutions
The password-protected 2018 Lone Wolf scenario can be downloaded from here:
Lone Wolf Teacher’s Guide.zip
If you are having trouble downloading and decompressing this file with a Macintosh using Safari, you should try to download and decompress it from the command line.
The teacher’s guide uses the same password as the other solutions available on this website. Please remember that the password to the teacher’s guide is only available to faculty at accredited universities. Further information can be found at the web page obtaining solutions.
Obtaining Solutions – Digital Corpora
Solution packets for these scenarios are available as encrypted PDF and ZIP files, including:
M57-Patents
Nitroba PCAP Solutions
M57-Jean
2012 National Gallery DC Attack Scenario
2018 Lone Wolf Scenario
2019 Narcos Scenario
Please note: the decrypt password is only available to faculty at accredited educational institutions, trainers and researchers working for the US Government, and (in some cases), forensics educators and researchers at partner governments. Students should not submit a request for solutions: such requests will be ignored. Instead, students should ask their faculty members to submit.
The decrypt password is provided to faculty members teaching courses in digital forensics as accredited educational institutions. Solutions should not be posted or provided to students: they are only to be used for course development and grading.
All solutions currently use the same password.
To get the solution password, please contact us with the WordPress contact form and provide
your full name
your phone number
an official web page that describes your course and clearly indicates your email address.
How many students and at what level (undergraduate, graduate) will be using the materials.
Whether or not we can put you on an announcement-only mailing list regarding new teaching materials we are developing.
Thank you!
2009 M57-Jean – Digital Corpora
The M57-Jean scenario is a single disk image scenario involving the exfiltration of corporate documents from the laptop of a senior executive. The scenario involves a small start-up company, M57.Biz. A few weeks into inception a confidential spreadsheet that contains the names and salaries of the company’s key employees was found posted to the “comments” section of one of the firm’s competitors. The spreadsheet only existed on one of M57’s officers, Jean.
Jean says that she has no idea how the data left her laptop and that she must have been hacked.
You have been given a disk image of Jean’s laptop. Your job is to figure out how the data was stolen, or if Jean isn’t as innocent as she claims.
Note: Solutions to this problem have been widely distributed on the Internet, so this assignment should only be used for self-study, and not for academic credit.
Materials:
Jean’s disk in EnCase E01 format:
nps-2008-jean.E01
nps-2008-jean.E02
(Note: nps-2008-jean is a multi-volume Expert Witness file. You need to download both of the files and put them in the same directory, or else you will not be able to process the disk image.)
Exercise Slides: 
M57-Jean.ppt (Microsoft PowerPoint format)
M57-Jean.key (Apple keynote format)
M57-Jean.pdf (Adobe Acrobat format)
Many students have had problems accessing these files with Autopsy. There is nothing wrong with these files or with Autopsy. Students: If you are having problems, you need to speak with your professor.
Solutions:
The solution is distributed as an encrypted PDF file:
M57-Jean_Solution.pdf
Please see our note on obtaining solutions.
Under Development – Digital Corpora
This page tracks the various corpora that are under development.
Scenarios
nitroba university – packets eavesdropped at a NAT box of a dorm room. One of the individuals inside the dorm room has been harassing a professor at Nitroba University.
m57-exfiltration — A confidential document from the M57 startup was posted on the website of a competitor. You have the hard drives of various individuals in the firm and you need to figure how it got out.
m57-patents — Objectionable material was found on a computer that was sold on Craigs List. The computer was traced back to the M57 startup. You need to determine how the objectionable material got there and who was responsible for it. This scenario contains several other problems.
Test Disks
We also have a number of test disks that are under development.
SQLite Forensic Corpus – Digital Corpora
As one of the most widely spread database systems in the world, SQLite is used on an immense number of computer systems. A vast number of software programs like browsers or smartphone apps are using SQLite3 databases to store application data. In many cases, such data is of high value during a forensic investigation. Therefore, various tools have been developed that claim to support rigorous forensic analysis of SQLite database files. Such claims are not supported by appropriate evidence, as standardized collections of databases were long missing. These can be leveraged by the forensic community for purposes like testing, validating, comparing and improving such tools.
We present a standardized corpus of SQLite files that can be used to evaluate and benchmark analysis methods and tools. The corpus contains databases which use special features of the SQLite file format or contain potential pitfalls to detect errors in (not only forensic) software. The corpus has also been enhanced by anti-forensic aspects within specifically crafted databases. These do not necessarily conform to the SQLite file format specification and can thus be used to additionally challenge the extraction and recovery routines of (forensic) tools. We call the database collection “SQLite Forensic Corpus” and donate it into the public domain.
A link to the corpus can be found here
The Articles linked to the corpus:
Sebastian Nemetz, Sven Schmitt and Felix Freiling, “A standardized
corpus for SQLite database forensics”, Digital Forensics Research
Workshop Europe, 2018
https://www.sciencedirect.com/science/article/pii/S1742287618300471
Sven Schmitt, “Introducing Anti-Forensics to SQLite Corpora and Tool
Testing”, IMF 2018, 11th International Conference on IT Security
Incident Management & IT Forensics
https://www.computer.org/csdl/proceedings-article/imf/2018/663200a089/17D45WnnFUl
Bulk Extractor News and Downloads – Digital Corpora
File bulk_extractor-1.3.1.zip contains the source code for bulk_extractor v1.3.1.Â bulk_extractor is a C++ program that scans a disk image, a file, or a directory of files and extracts useful information without parsing the file system or file system structures.Â bulk_extractor is typically downloaded on a Fedora system and compiled or cross-compiled to Linux, Mac, or Windows using autotools.Â Please seeÂ https://github.com/simsong/bulk_extractor/wiki/Introducing-bulk_extractor.
BEViewer.jar is an executable bulk_extractor viewer user interface.
Bulk Extractor ViewerÂ (BEViewer) provides a graphical user interface for browsing features that have been extracted via theÂ bulk extractorÂ feature extraction tool.Â Please seeÂ https://github.com/simsong/bulk_extractor/wiki/BEViewer.
be_installer-1.3.exe is a Windows installer for installing bulk_extractor and BEViewer v1.3 on a Windows system.
bulk_extractor.pdf, “Digital media triage with bulk data analysis and bulk-extractor,” discusses how the bulk_extractor tool is effective in providing bulk data analysis.
2012-08-08 bulk_extractor Tutorial.pdfÂ describes how to use the BEViewer tool.Â Although some of the parameters for running bulk_extractor have changed, the majority of the tutorial remains current..
Source: The information above and links were received from Bruce Allen <bdallen@nps.edu>, Naval Postgraduate School
See other bulk_extractor downloads here:Â https://digitalcorpora.org/downloads/bulk_extractor/
CIRCL Forensics Exercises – Digital Corpora
CIRCL Forensics Exercises are little challenges developed for and during the CIRCL Forensics Trainings, and for workshops or presentations. Usually you will find a PDF with the slides and the solution inline, next to a disk image with the challenge itself.
Wiped Disk Image:
https://downloads.digitalcorpora.org/corpora/drives/circl-2023-wiped/
Recovering data from a wiped disk sounds impossible. But wiping a ‘big’ disk would take time. If the adversary is not patient and interrupts the wiping process after some limited time, there is a good chance to recover some plain data, or even complete file system structures and/or partitions. Goal of this exercise is, to recover data from a partially wiped disk image.
This is a manual approach, to learn how to analyze data on byte level.
To be continued
Forensic Innovations, Inc., analyzes the million file corpus – Digital Corpora
Forensic Innovations, Inc., makers of File Investigator TOOLS, has performed an analysis of the 986,278 files in the “1 million file corpus”. (13,722 files in the corpus were removed earlier this year because they were from California State Government web servers that were in the .gov domain and mistakingly collected as part of the original collection effort.)
We would like to thank Forensic Innovations for their work in support of this project. We have made available their summary report and will be making available their file-by-file analysis as soon as we deploy an appropriate database on this website.
Simple Statistical Report of the 1 million file corpus.
Hash Codes – Digital Corpora
The following post is now obsolete. The file frequent_hashcodes_and_paths_rdc.xml has been removed from the corpus as explained in a more recent post. Please see:
https://digitalcorpora.org/archives/391
Deprecated Post from Mar 29, 2013 @ 13:13
The file frequent_hashcodes_and_paths_rdc.xml contains SHA1 hashcode and path data derived from the Real Drive Corpus collected by the DEEP Project at the U.S. Naval Postgraduate School. The file provides two kinds of data useful to forensic investigators: (1) SHA1 hashcodes that occurred for undeleted files on at least five different drives in the corpus but did not occur in the National Software Reference Library (https://www.nsrl.nist.gov). These are likely to indicate files uninteresting and excludable in most forensic investigations. File sizes and names are also given. (2) Path names (file name plus all directories) for paths that occurred on at least twenty different drives in the corpus on undeleted files. These usefully supplement the hashcodes in indicating recurring files uninteresting for investigators. However, occurrences of these files could include viruses and other malware, or could be hiding illegal content although it is unlikely.
Read more … https://digitalcorpora.org/corp/hashes/nus-deidentified/README-frequent-hashcodes-and-paths-rdc.txtDownload XML File (HAS BEEN REMOVED): https://digitalcorpora.org/corp/hashes/nus-deidentified/frequent-hashcodes-and-paths-rdc.xml (102 MB)
Digital Corpora Announcement List – Digital Corpora
Leave a Reply 
Your email address will not be published. Required fields are marked *
Comment * 
You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> 
 <strike> <strong> 
Save my name, email, and website in this browser for the next time I comment.
Notify me of follow-up comments by email.
Notify me of new posts by email.
Nitroba University Scenario Available – Digital Corpora
Leave a Reply 
Your email address will not be published. Required fields are marked *
Comment * 
You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> 
 <strike> <strong> 
Save my name, email, and website in this browser for the next time I comment.
Notify me of follow-up comments by email.
Notify me of new posts by email.
M57-Jean Scenario Posted – Digital Corpora
Leave a Reply 
Your email address will not be published. Required fields are marked *
Comment * 
You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> 
 <strike> <strong> 
Save my name, email, and website in this browser for the next time I comment.
Notify me of follow-up comments by email.
Notify me of new posts by email.
MySQL tables for NIST NSRL RDS 2.26 posted – Digital Corpora
Ever want to have SQL access to the NIST RDS but didn’t want to spend a month building the MySQL tables? Well, we did too… So we took one of our 8-core, 32GB servers, imported all of the NSRL, and then put a tar file of the tables available for download on this server.
To use these files just download https://downloads.digitalcorpora.org/corpora/files/nist-nsrl-rds226.tar.bz2 and put the files in your MySQL data directory. You’ll be up-and-running in no time.