hemu1438/forensicsgpt-training-data huggingface dataset:
Getting started in forensic research -
Interested in getting involved in computer forensics research? Here's how to start.
Recommended Reading
Read the proceedings for the past four years of the DFRWS conference.
Review the proceedings from the past few years of the IEEE/SADFE (Systematic Approaches to Digital Forensics Engineering) workshops.
Review the IFIP Working Group 11.9 on Digital Forensics website and look at the proceedings from the past conferences (unfortunately, you can't download the papers and the book costs more than \$100, but if you see something interesting it can usually be requested via interlibrary loan) (Some higher education libraries subscribe to SpringerLink which makes full text of these proceedings available to students and faculty as part of the school subscription)
IFIP WG 11.9 publications
Search for interesting forensic terms at the ACM Digital Library and CiteSeer
Review the Sleuth Kit Website. In particular, review the issues of The Sleuth Kit Informer and download a copy of Sleuth Kit for your computer.
Recommended Mailing Lists
sleuthkit-users
computer forensic tool testing (cftt)
Setting up a C++ development environment
Many people working in forensics find it useful to be able to compile their tools from source code. Most of the tools compile on Linux, Mac, and within the Cygwin environment under Windows.
Because all of these tools build upon one another, it is important to compile and install them in the order specified below.
Download a copy of libewf and install it on your computer. This will allow your forensic tools to read and process EnCase E01 disk images.
Download a copy of Sleuthkit and install it. Sleuth Kit is the basic open source computer forensics tool that allows the extraction of files from disk images. You can use it to recover deleted files.
If you are interested in doing file recovery, you may also wish to explore:
Sleuth Kit, above
PhotoRec, a file carver.
Adroit Photo Recovery, a commercial photo recovery tool that's pretty amazing.
If you want to experiment with automated computer forensics research, try these:
Bulk Extractor, a program from the Naval Postgraduate School that searches a disk image for email addresses and prints a histogram.
fiwalk, a program that processes a disk image and outputs an XML or ARFF file containing information about all of the file system metadata. fiwalk is now part of Sleuth Kit.
Exercises for the Reader
Download nps-2009-canon2-gen6 from the Digital Corpora website and try to recover as many files as you can. Some of the JPEGs can only be found using file carving, and some can only be found with fragment recovery file carving.
Can you determine when the photos were taken?
Can you determine where the photos were taken?
Can you determine the username of the person who took the photos?
Can you determine the clock offset of the camera from real time?
Download nps-2009-ubnist1 and find the government documents that were stored on the USB device.
Ocfalib api -
The OcfaLib API is a C++ API that is meant for usage by modules in the Open Computer Forensics Architecture. The C++ class ocfa::module::OcfaModule defines the base class of what all OCFA modules are derived. Depending on the type of module, the ocfa::facade namespace provides different convenient subclasses. The OcfaModule class defines a virtual function named ProcessEvidence that the final module class must implement. A typical module implementation implements a subclass by implementing a constructor and a ProcessEvidence method. The main function of the module program will create an instance of the module class and call the run method on the object. The module will connect and register itself to the OCFA Anycast Relay and thus be connected into the Open Computer Forensics Architecture. For each piece of evidence data the module receives, the ProcessEvidence method will get invoked. The implementation of the ProcessEvidence method can depending on the type of facade used as baseclass can:
Gain read access to the input evidence data.
Use its own private workdir for derived and temporary data
Derive evidence from the input evidence.
Access meta data created by other modules.
Add additional metadata to the input evidence or child evidences.
For a simple example of a module that derives data from data look at the tar module in OcfaModules/dissector/tar. For a simple example of a module that extracts metadata from data look at the pgp module in OcfaModule/extractor/pgp.
Next to simple OCFA modules, the ocfa::treegraph namespace provides an aditional sub API for building treegraph loadable modules, that can be used to create modules that map evidence data to a tree graph of data and meta data.
The only API available currently is the C++ API. Work is currently being done to also create a Java API and plans exist to build a Perl API to OCFA in the future.
See Also
Open Computer Forensics Architecture
M3 bitlocker recovery -
M3 Bitlocker Recovery is a Free Bitlocker recovery software to recover data and decrypt data from Bitlocker encrypted volume created on Windows Vista/7/8/8.1/10 and Windows Server 2012/2008 as long as you provide the original Bitlocker recovery key/password or encryption password.
Features
Recover deleted files from Bitlocker encrypted volume.
Recover data from formatted Bitlocker encrypted volume.
Recover deleted or lost Bitlocker encrypted volume.
Decrypt data from damaged, corrupted or inaccessible Bitlocker encrypted volume.
Recover data after Bitlocker Disk Encryption failed, interrupted, stuck or freeze.
Recover data after Bitlocker drive decryption failed, interrupted, stuck or freeze.
Recover data after Bitlocker drive encryption is not completed.
Supported File System
M3 Bitlocker Recovery supports FAT and NTFS file systems.
Supported Operating System
M3 Bitlocker Recovery supports Windows XP, Windows Vista, Windows 7, Windows 8, Windows 8.1, Windows 10 and Windows Server 2003, Windows Server 2008, Windows Server 2012.
See Also
M3 Data Recovery
M3 RAW Drive Recovery
M3 Partition Recovery
M3 Data Recovery for Mac
M3 Bitlocker Loader for Mac
External Links
Download Link
M3 Bitlocker Recovery
Gmail header format -
Because Gmail is a web based application and can be changed at any time, the information in this article may not reflect the current state of Gmail headers. In general Gmail headers have a DomainKeys Identified Mail signature line that contains a signature for the message in question. These lines appear above the standard Message-ID fields. These signatures are of the format:
DKIM-Signature: a=rsa-sha1; c=relaxed/relaxed; d=gmail.com; s=beta; h=domainkey-signature:received:received:message-id:date:from:to:subject:mime-version:content-type; b=OITvzFGKQQUjywUQB7U8dQypDAeOGqBIhfcb8VKioP2UU5P2aJL3l2adoyRqSp9h/Fo9A6wY5EIRsfaCWM9ge+EzCob/ 4p85jcEn3uW8dpRyBFQXMuK2q0RMIk3FznrXAM4W5FvoJIPP04qgXErar+/hZq03vEUIErV1v6p2Fy4= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=beta; h=received:message-id:date:from:to:subject:mime-version:content-type; b=oC+hlWhBboQ+RlsKCL4r2pQxpgKRM9iUgCBmw9wZqlEcxj+A3q+fJkDXgLKmI1twfvTHj7GQ3HDzSLzw982UD +CPh1bPQxkhNbylUBRtwpoFeixIk7OmR2YE1iYrYpQXf3dEcXNfKs7ffoeY18plJNJG0S8RRmXLaR6XqXFVUoo= 
Note that some of the Received lines will contain hosts with IP addresses like 10.x.x.x. These addresses are non-routable but part of the Gmail system. The remaining headers look like:
Message-ID: <f6a363400703050910y7d591d42raf015fcef16f95ea@mail.gmail.com> Date: Mon, 5 Mar 2007 09:10:41 -0800 From: UserName <address@gmail.com> To: OtherUserName <address@system.com> Subject: Subject Line MIME-Version: 1.0 
Message Id Field
Although the message id format is not completely understood, it contains the date and time the message was sent, in pacific time, starting at the ninth digit, in the form YYMMDDHHMM. In the example above, the time/date is 0703050910, 0910 PDT on 5 Mar 2007. This has been confirmed by experimental testing. If you are trying to determine if a message has been forged, you should verify that the time in the Message Id field matches the time in the Date field for the message. Don't forget that the time in the Date field should be in the Sender's local time and the Message Id will be in Google's local time, US Pacific Time.
See Also
Using message id headers to determine if an email has been forged
External Links
Gmail
DKIM Signatures
M3 data recovery for mac -
M3 Data Recovery for Mac is a free data recovery software to recover deleted files or lost files in Mac computer.
Features
1. Recover files deleted by pressing Command + Del or emptied from Trash.
2. Recover data from formatted partition or inaccessible partition.
3. Recover data from deleted or lost partition due to partition loss, partition deleting, and partition error.
4. Recover lost files from hard drive, external hard drive, USB drive, memory card, SD card, CF card and other devices.
Supported File System
M3 Data Recovery for Mac supports HFS, HFS+, FAT32 and HFSX file systems.
Supported Operating System
M3 Data Recovery for Mac supports Mac OS X 10.10 (Yosemite), 10.9 (Mavericks), 10.8 (Mountain Lion), 10.7 (Lion).
See Also
M3 Bitlocker Loader for Mac
M3 Bitlocker Recovery
M3 RAW Drive Recovery
External Links
Download Link
M3 Data Recovery for Mac
Ontrack data eraser -
Reportedly this program places a 10 digit number at the beginning of each sector followed by short hand date/time stamp. The date/time stamp is consistent throughout all sectors and reflects the time the wipe was started. The date/time stamp was followed by any specified character, such as the letter "E".
0000000000 1 JLY 2004 11:15:44 EEEEEEEEEEEEEEEEE... 0000000001 1 JLY 2004 11:15:44 EEEEEEEEEEEEEEEEE... 0000000002 1 JLY 2004 11:15:44 EEEEEEEEEEEEEEEEE... ....
Gmx header format -
Content-Type: text/plain; charset="us-ascii" Date: Sun, 29 Jul 2007 13:08:52 +0200 From: Username <username@sendinghost.com> Message-ID: <20070729110852.111111@gmx.net> MIME-Version: 1.0 Subject: header test To: Username <username@receivinghost.com> X-Authenticated: #33333333 X-Mailer: WWW-Mail 6100 (Global Message Exchange) X-Priority: 3 X-Provags-ID: G11D2DsdDDkF13DrwAc8wFFGyftuH1D+5jsXgvVepqK7XQlVHKHiu SekD1HHRJ3gH+9uX8afy+MN4KLoamn6RTgjE== Content-Transfer-Encoding: 7bit 
Message-ID format:
[UTC-date].[login count]@[host] 
- [login count] it looks like this field is count of logins on gmx.net for all users. Till next login this part will be constant. X-Authenticated: describe personal user id. X-Provags-ID is unknown.
Linux user accounts -
User's Configuration
/etc/passwd: Contain basic users data. UID 0 are reserved for root, 1-99 reserved for predefined accounts ,and 100-999 reserved for system accounts/groups. Format 'username:x:UID:GID:Userinfo:homedir:shell' /etc/shadow: Contain the password hashes of the users. Can be cracked using 'John The Ripper' or 'HashCat /etc/sudoers: controls who can run what commands as what users on what machines /etc/group: Contains groups memberships 
Backups
Files that end with the '~' character (passwd~) are the previous version of the original file (passed). 
Home
Home folder of each user is located under '/home//' and for root home folder is '/root/'.
Artifacts: .bash_hisotry, App specific configuration files and ssh configurations
M3 partition recovery -
M3 Partition Recovery is a professional partition recovery software to recover deleted partition, recover lost partition, recover RAW drive, recover damaged or corrupted drive.
Supported File System
M3 Partition Recovery supports FAT and NTFS file systems.
Supported Operating System
M3 Partition Recovery supports Windows XP, Windows Vista, Windows 7, Windows 8, Windows 8.1, Windows 10 and Windows Server 2003, Windows Server 2008, Windows Server 2012.
See Also
M3 RAW Drive Recovery
M3 Bitlocker Recovery
M3 Data Recovery for Mac
M3 Bitlocker Loader for Mac
External Links
Download Link
M3 Partition Recovery
M3 raw drive recovery -
M3 RAW Drive Recovery is a RAW drive partition recovery and data recovery software to solve RAW File System and RAW drive issue.
RAW Drive Partition Recovery
Partition recovery function built-in M3 RAW Drive Recovery is used to fix RAW drive, repair corrupted or damaged drive and convert RAW to NTFS/FAT32 file system without data loss.
RAW Drive Data Recovery
Data recovery function built-in M3 RAW Drive Recovery is used to recover data from RAW hard drive, external hard drive, SD card, memory card, USB flash drive, etc.
Supported File System
M3 RAW Drive Recovery supports FAT and NTFS file systems.
Supported Operating System
M3 RAW Drive Recovery supports Windows XP, Windows Vista, Windows 7, Windows 8, Windows 8.1, Windows 10 and Windows Server 2003, Windows Server 2008, Windows Server 2012.
See Also
M3 Bitlocker Recovery
M3 Partition Recovery
M3 Data Recovery for Mac
M3 Bitlocker Loader for Mac
External Links
Download Link
M3 RAW Drive Recovery
Open computer forensics architecture -
The Open Computer Forensics Architecture (OCFA) is a modular computer forensics framework built by the Dutch National Police Agency. The main goal is to automate the digital forensic process to speed up the investigation and give tactical investigators direct access to the seized data through an easy to use search and browse interface.
The architecture forms an environment where existing forensic tools and libraries can be easily plugged into the architecture and can thus be made part of the recursive extraction of data and metadata from digital evidence.
The Open Computer Forensics Architecture aims to be highly modular, robust, fault tolerant, recursive and scalable in order to be usable in large investigations that spawn numerous terabytes of evidence data and covers hundreds of evidence items.
Modules in OCFA for reasons of fault tolerance are processes. The basic OcfaLib API makes it possible and relatively easy to build an OCFA module out of any data processing library or tool. OCFA comes with numerous such modules that are mostly wrappers around libraries like libmagic or tools such as those found in the The Sleuth Kit.
The 2.2 version of OCFA (released April 2009) makes the previously internal OCFA treegraph API available for OCFA module development. The OCFA treegraph API allows more advanced dissectors that produce data and meta-data for a treegraph representation of an input file. The OCFA treegraph API also allows dissectors that are programed to be CarvFs aware to use zero storage carving.
Communication between modules within OCFA is governed by a two layered communication infrastructure as provided by OCFA. At the lowest layer is a messaging system with at is center the OCFA AnyCast-relay. The Anycast Relay provides the facilities of module crash resistance, distributed processing load balancing and flow control. At a higher level of communication, the OCFA XML Router provides for the routing of individual pieces of evidence through the most appropriate tool chain for its particular type of content.
Although OCFA contains a rudimentary user interface, most of its power is in the backend architecture. The last and final module in the tool chain of any evidence will be the OCFA Data Store Module. This module processes the evidence XML (that contains all of the evidence data its meta data) and stores relevant parts into a postgesql database. Extending the apache based user interface with interfaces for your own case bound queries is something that should proof very useful in most investigations.
Development and maintenance of OCFA by the Dutch National Police has been discontinued in 2012. The latest version of the orphaned code is available as the seperate github repositories ocfaLib , OcfaArch, OcfaJavaLib, OcfaModules and OcfaDocs on the Dutch National Police Github page.
External Links
Project site
Linux Magazine article on OCFA
Open Computer Forensic Architecture a Way to Process Terabytes of Forensic Disk Images
Open webmail header format -
From: "Username" <username@sendinghost.com> To: "Username" <username@receivinghost.com> Subject: Header test Date: Thu, 2 Aug 2007 20:54:36 +0400 Message-Id: <20070802165414.M43479@sendinghost.com> X-Mailer: Open WebMail 2.50 20050106 X-OriginatingIP: 255.255.255.255 (username@sendinghost.com) MIME-Version: 1.0 Content-Type: text/plain; charset=koi8-r 
Message-ID format:
openwebmail/openwebmail-tool.pl # fake a messageid for this message my $fakedid = ow::datetime::gmtime2dateserial().'.M'.int(rand()*100000); $fakedid="<$fakedid".'@'."${$config{'domainnames'}}[0]>";
List of mua header formats -
The following pages defines the headers added to each message by the mail user agent:
Apple Mail Header Format
Eudora Header Format
Evolution Header Format
Gmail Header Format
GMX Header Format
Horde IMP Header Format
Hotmail Header Format
iPhone Mail Header Format
Microsoft Mail Header Format
Mutt Header Format
Open WebMail Header Format
Outlook Header Format
Outlook Express Header Format
Pine Header Format
The Bat! Header Format
Thunderbird Header Format
Yahoo! Mail Header Format
See Also
Email Headers
Openssl -
OpenSSL is an open source software system that provides the following:
Forensic-grade implementations of the most widely used hash functions.
Symmetric cryptographic functions
Asymmetric cryptographic function
Certificate management functions
A complete S/MIME implementation
A complete SSL/TLS implementation
OpenSSL is interesting for forensic practitioners and developers because it provides a basic toolkit for building software, and because the higher-level certificate management functions give you an easy way to decode the contents of certificates that are used to secure computer systems.
This web page contains step-by-step instructions on using OpenSSL from the command line to perform specific tasks. There are a lot of online OpenSSL guides and we'll try to link to some of them from here. But this page is a handy reference just the same.
File Extensions
OpenSSL doesn't care what you use for file extensions. However, the following extensions to seem to be commonly used:
File Extension Meaning 
.pem	can contain a private key, public key, or certificate signing request.	
.crt	Windows file extension for a .pem file.	
.p12	a PKCS12 file, which contains a private key and a certificate, encrypted for transport with a passphrase. This is the format that Windows and MacOS like to import	
Conversion
convert pem to pkcs12:
% openssl pkcs12 -export -in mpage.crt -inkey mpage.key -out mpage.p12 -name 'MPage Signing Key'
convert pkcs12 to pem, putting both private key and certificate in the same file
% openssl pkcs12 -in mpage.p12 -out mpage.pem
The same, but with no encryption of file
% openssl pkcs12 -in mpage.p12 -out mpage.pem -nodes
Decrypt a PEM file private key:
% openssl rsa -in newreq.pem -out key.pem
Print the contents of a certificate
% openssl x509 -in mpage.pem -text
Input the PKCS12 file and output a key file and a cert file:
openssl pkcs12 -in slg.p12 -out slg.key -nocerts -nodes openssl pkcs12 -in slg.p12 -out slg.pem -nokeys -nodes
Making Certificates
To make certificates all in one step:
openssl req -new -x509 -nodes -out imapd.pem -keyout imapd.pem -days 3650
Make a certificate request for a CA to sign:
openssl req -newkey rsa:1024 -keyout mycert.key -out mycert.csr -nodes
To make an RSA key and then use the key to make the certificate signing request:
openssl genrsa -out myfile.key 1024 openssl req -new -key myfile.key -out myfile.csr
Get a certificate from an SSL server
openssl s_client -connect www.nitroba.com:443
Viewing Certificates
openssl x509 -in ssl.crt-text
S/MIME
to sign an outgoing mail:
from_email = `openssl x509 -email -in certfile.pem -noout` x509_subject = `openssl x509 -subject -in certfile.pem -noout` openssl smime -from %s -to %s -subject %s -sign -inkey file -signer %s -in tempfile.txt extra
See Also
https://www.oreilly.com/radar/?page=4
Hashdb -
hashdb is a tool for finding previously identified blocks of data in media such as disk images. hashdb provides the following:
The hashdb tool used to create hash databases, import block hashes, provide lookup services, and otherwise manage block hash databases.
The hashdb library that allows other programs to create hash databases or scan for block hashes. The hashid scanner is a bulk_extractor plugin that uses libhashdb to search for previously identified blocks of data.
See Also
Bulk Extractor, the feature extraction tool.
External Links
Project site
Optimistic decompression -
Optimistic decompression is a digital forensics technique in which each byte of an input buffer is examined for the possibility of compressed data. If data is found that might be compressed, a decompression algorithm is invoked to perform a trial decompression. If the decompressor does not produce an error, the decompressed data is processed. The decompressor is thus called optimistically---that is, with the hope that it might be successful.
Optimistic decompression is used as a tool to validate JPEGs that are produced by file carving. The approach can also be used for fragment reassembly carving by attempting to combine different combinations of fragments, looking for a combination that produces neither decompression errors nor visual discontinuities. This is the approach used by the Adroit file carver.
Outlook express header format -
Version 6 of Outlook Express running on Microsoft Windows generates headers in the format:
Message-ID: <000f10c7183d$abe4d510$6031a8c0@hostname> From: "Username" <username@sendinghost.com> To: "Username" <username@receivinghost.com> Subject: Testing Date: Wed, 4 Apr 2007 14:11:45 +0100 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2800.1807 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1807 
Message ID Field
The Message-ID have 4 parts: [hex time]\$[random?]\$[hw-hash?]@[hostname]
Microsoft exchange server -
Microsoft Exchange Server stands as the most suitable emailing server platform. Developed and owned by Microsoft; the very first release of Exchange, i.e. Exchange Server 4.0 was made in March 1996. Exchange Server can be considered as a centralized repository that keeps hold of all the emails online. From small and medium - sized firms to large multinational corporations; Exchange Server is being effectively used and leverages supports such as sending and receiving of emails, calendaring, creation of distribution lists, handling voicemail transcriptions and many more.
Why Exchange Server?
Microsoft Exchange Server stands as the most outstanding and cost – effective solution delivering highly secure B2C and B2B emailing solution.
Equipped with DLP (Data Loss Prevention) capabilities, that excels in protecting sensitive data.
Allows technocrats to perform in – place eDiscovery across multiple environments such as Exchange, SharePoint, and Lync.
Use Exchange Hybrid, a platform that allows you to traverse your emails on – premise and meet your business needs.
Features
Microsoft Exchange Server 2013 was released on October 11, 2012. The various features that make Exchange Server an indispensable product are: -
DAG (Database Availability Groups). DAG is a Mailbox sever high availability and site resilience framework. Once any server is added to DAG, it eventually gets synchronized to provide automatic recovery from failures.
CAS (Client Access Server). CAS arrays are being used by Exchange Server to provide a highly enhanced Client Server Access with dedicated availability.
Offline Support in OWA. Users can work offline and all the actions automatically get synchronized when the connection is regained.
Enhanced Outlook Web App. Extended optimized UI support for mobile phone, tablet, and desktop browsers.
Disk Support Up To 8 TB. Multiple databases are being used in conjunction with DAG.
In – built anti – malware. Customizable feature of usage of anti – malware provided for use, if required.
In – Place eDiscovery. Crucial and sensitive data can now easily be searched and carved in.
Shadow Redundancy. “Shadow Redundancy” mechanism protects the email messages in the state when they are available in the transit.
Reference Links
Extensible Storage Engine (ESE) Database File (EDB) format
External Links
Wikipedia on Microsoft Exchange Server