hemu1438/forensicsgpt-training-data huggingface dataset:
Nasa office of the inspector general -
The Office of the Inspector General (OIG) is the investigative arm of the United States National Aeronautics and Space Administration. They have a strong computer investigative program that is quite adept with Linux based systems.
See Also
NASA OIG has generously allowed us to publish their policy for the preservation and analysis of digital evidence by computer forensic specialists and special agents. In their words, "It's our hope that this document will help others, particularly law enforcement organizations, with policy development in this growing area of importance. We also hope that others will consider sharing portions of their policy and procedures as they are developed and continuously refined."
External Links
Official website
The Law Enforcement and Forensic Examiner's Introduction to Linux, by SA Barry Grundy
Mailing lists -
There are a number of mailing lists of interest to Digital Forensics Practitioners. Below is a listing of some more well known lists. It is not all-inclusive.
American Academy of Forensic Science, Digital and Multimedia Sciences Section - Subscription requires moderator approval. The archive is only available to list members. This list supports the Digital and Multimedia Sciences section in the AAFS. Subscribe by sending an email to listserv@lists.mitre.org with message body containing SUBSCRIBE AAFS-DIGITAL-MULTIMEDIA-LIST.
Computer Forensics Investigators Digest (CFID) - Subscription requires moderator approval. The archive is only available to list members. Subscribers must meet the following criteria: Be a full-time member of either law enforcement or corporate security tasked with the investigations of high-technology crime and/or computer forensics; Not have ever worked for the defense in a criminal case (ethical defense work excepted on a case by case basis); Not be a primary vendor of forensic products (exceptions to this are personnel who retire and then offer their utility suites commercially). For subscription information, send request to jnj AT infobin.org
Computer Forensics Tool Testing (cftt@yahoogroups.com) - Subscription requires moderator approval. The archive is only available to list members.
High Technology Crime Consortium Mailing List - The HTCC mailing list is for members only. The archive is only available to list members.
High Technology Crime Investigation Association Mailing List - The HTCIA mailing list is for members only. The archive is only available to list members.
MacIntosh OS: Forensics (macos_forensics@yahoogroups.com) - The archive is only available to list members.
Linux logical volume manager (lvm) -
The Linux Logical Volume Manager, is commonly abbreviated to LVM. Although LVM can used for other Logical Volume Management variants as well.
Not all forensic tools have support for Linux Logical Volume Manager (LVM) volumes, but most modern Linux distributions do.
Forensic analysis
The metadata area of a LVM Physical Volume can contain multiple versions of metadata section that contains the LVM Volume Group definitions, including a creation date and time value.
Mounting an LVM
Mounting an LVM from an image
If you have an image mount the LVM read-only on a loopback device (e.g. /dev/loop1) by:
sudo losetup -r -o $OFFSET /dev/loop1 image.raw 
Note that the offset is in bytes.
If you need to write to the image, e.g. for recovery, use xmount to write the changes to a shadow file.
sudo xmount --in dd --cache sda.shadow sda.raw image/ 
You can then safely mount the LVM in read-write mode (just omit the -r in the previous losetup command).
To remove this mapping afterwards run:
sudo losetup -d /dev/loop1 
To scan for new physical volumes:
lvm pvscan 
You cannot unmount an active volume group. To detach (or deactivate) the volume group:
vgchange -a n $VOLUMEGROUP 
Where \$VOLUMEGROUP is the corresponding name of the volume group
The individual volume devices are now available in:
/dev/mapper/$VOLUMEGROUP-$VOLUMENAME 
Mounting an LVM from a device
To list the Volume Groups (VG) run:
pvs 
To list information about a Volume Group (VG) run:
lvdisplay $VOLUMEGROUP 
The field "LV Name" provides the volume name
To make the volume group known to the system (only if it was previously exported using vgexport command, note that vgimport will alter the data in the LVM2 header):
vgimport $VOLUMEGROUP 
And activate the volumes in the volume group (sometimes you need to run vgscan first):
vgchange -a y $VOLUMEGROUP 
Warning: LVM driver will synchronize all physical volumes if they are out of sync after the command above is issued
The individual volume devices are now available in:
/dev/mapper/$VOLUMEGROUP-$VOLUMENAME 
These now can be analyzed with e.g. a tool like the The Sleuth Kit or loop-back mounted.
To read-only loop-back mount an individual volume:
mount -o ro,loop /dev/mapper/$VOLUMEGROUP-$VOLUMENAME filesystem/ 
Also see
File Systems
External Links
Wikipedia article on Logical Volume Manager
LVM Howto, by The Linux Documentation Project
LVM2 Resource Page
The Linux Logical Volume Manager, by Heinz Mauelshagen and Matthew O'Keefe
LVM cheatsheet, by RedHat
Unix/Linux Administration Logical Volume Management Guide, by Wayne Pollock, 2005
Logical Volume Manager ‐ Software RAID, by Proneer, March 30, 2010
Tools
libvslvm
Online resources -
There are lots of websites that can provide valuable information for forensic investigators.
WHOIS Queries
The WHOIS Service can be used to find the owner of a domain. Sometimes this is only sufficient to find the registrar for a domain, but even that is a start.
WHOIS-Search.com
Web Page Archives
Web page archives can give the investigator a look at what a web page used to look like. The most well known is the Google cache, but here are some others:
The Internet Archive's WayBack Machine can produce a nicely formatted page showing how a website looked on certain dates. For example, the history of yahoo.com's homepage gives a nice history lesson. The machine records both main pages and subpages.
Oxygen forensic suite -
Current version
	
Version Number: 6.5
Date Released: 19 September 2014
	
Recent changes
	
Extract data from iPhone 6 and iPhone 6 Plus
Acquire contacts and calls from Windows Phone devices
Live BlackBerry 10 acquisition
Import Nokia NBU backuf files
	
Oxygen Forensic Suite is a mobile forensic software for logical analysis of cell phones, smartphones and PDAs developed by Oxygen Software.
The suite can extract device information, contacts, calendar events, SMS messages, event logs, and files. In addition, the vendor claims the suite can extract metadata related to the above.
As of December 2014 the suite supported more than 8,400 devices, including Nokia, Apple iPhone series, Apple iPod Touch, Apple iPad, Vertu, Sony Ericsson, Samsung, Motorola, Blackberry, Panasonic, Siemens, HTC, HP, E-Ten, Gigabyte, i-Mate, Chinese (Mediatek) Phones and other mobile phones. The suite also supports devices running Symbian OS, Windows Mobile 5/6, Microsoft Windows Phone 8 and Android OS devices.
Forensic Soundness
The suite access devices using advanced proprietary protocols. Some devices like smartphones require an Agent installation. Of course, installing software onto the device being examined will impact the forensic soundness of the investigation. If such information is not obtainable by other means and the impact is documented, it may still be admissible under the Best Evidence Rule.
Previous Names
Oxygen Forensic Suite was previously marketed as "Oxygen Phone Manager II (Forensic Edition)".
External Links
Official website
P2pmarshal -
P2P Marshal is a program that helps an investigator discover and analyze file sharing software on a disk.
The Forensic Edition of P2P Marshal operates on a logically mounted drive (i.e., a restored image of a disk, mounted as D:, E:, etc. or with Mount Image Pro, EnCase's Physical Disk Emulator, or similar). The Field Edition can also analyze a live computer's disk. P2P Marshal is designed to run under Windows, though as a Java-based program, it should be able to run on other platforms.
When run, it first detects the presence of P2P client programs. Then, for each program detected, it presents various information, such as downloaded and shared files, peer servers, and log messages. For some clients, additional information may be displayed, such as host ID numbers, total runtime, and other parameters tracked by the client. P2P Marshal displays the information either on a per-user basis or for all users. It also provides an extensive search capability, produces customizable summary reports in PDF, HTML, and RTF formats, and maintains an audit log of all actions performed by the investigator.
The Forensic Edition must be installed on a computer; the Field Edition runs from a USB drive.
It currently supports LimeWire and several BitTorrent clients, and Ares, and detects the presence of KaZaA.
As of March 2010, the 2.1.1 version is current. The Forensic Edition of P2P Marshal is available to US law enforcement at no cost. ATC-NY distributes the tools to US law enforcement and for educational use. Cyber Security Technology distributes the tool commercially.
P2P Marshal was developed by ATC-NY through a US National Institute of Justice (NIJ) grant. The project was originally named File Marshal.
External Links
File Marshal: Automatic extraction of peer-to-peer data, by Frank Adelstein, Robert A. Joyce, DFRWS 2007
p2pmarshal.com
ATC-NY
Pdfinfo -
pdfinfo is a small utility which shows (parts of) the metadata stored in PDF files. It is part of the xpdf package.
Example
$ pdfinfo nsa-redact.pdf Title: Redacting with Confidence: How to Safely Publish Sanitized Reports Converted from Word to PDF Subject: I333-015R-2005 Keywords: word, pdf, redaction, metadata Author: SNAC Creator: activePDF DocConverter Producer: 5D PDFLib CreationDate: Tue Jan 10 11:15:26 2006 ModDate: Fri Jan 20 09:05:43 2006 Tagged: no Pages: 14 Encrypted: no Page size: 612 x 792 pts (letter) File size: 681417 bytes Optimized: yes PDF version: 1.3
The -meta command line option shows some more metadata (in XML.
Reiserfs -
Detecting ReiserFS in a forensics environment
Note: These are in little-endian format.
**Name** Size Description 
Block count 	4 	The number of blocks in the partition 	
Free blocks 	4 	The number of free blocks in the partition 	
Root block 	4 	The block number of the block containing the root node 	
Journal block 	4 	The block number of the block containing the first journal node 	
Journal device 	4 	Journal device number (not sure what for) 	
Orig. journal size 	4 	Original journal size. Needed when using partition on systems with different default journal sizes. 	
Journal trans. max 	4 	The maximum number of blocks in a transaction 	
Journal magic 	4 	A random magic number 	
Journal max batch 	4 	The maximum number of blocks in a transaction 	
Journal max commit age 	4 	Time in seconds of how old an asynchronous commit can be 	
Journal max trans. age 	4 	Time in seconds of how old a transaction can be 	
Blocksize 	2 	The size in bytes of a block 	
OID max size 	2 	The maximum size of the object id array 	
OID current size 	2 	The current size of the object id array 	
State 	2 	State of the partition: valid (1) or error (2) 	
Magic string 	12 	The reiserfs magic string, should be "ReIsEr2Fs" 	
Hash function code 	4 	The hash function that is being used to sort names in a directory 	
Tree Height 	2 	The current height of the disk tree 	
Bitmap number 	2 	The amount of bitmap blocks needed to address each block of the file system 	
Version 	2 	The reiserfs version number 	
Reserved 	2 		
Inode Generation 	4 	Number of the current inode generation. 	
The following is the start of the superblock of a 256MB reiserfs partition on an Intel based system:
00000000 66 00 01 00 93 18 00 00 82 40 00 00 12 00 00 00 f........@...... 00000010 00 00 00 00 00 20 00 00 00 04 00 00 ac 34 11 57 ..... ......¬4.W 00000020 84 03 00 00 1e 00 00 00 00 00 00 00 00 10 cc 03 ..............Ì. 00000030 08 00 02 00 52 65 49 73 45 72 32 46 73 00 00 00 ....ReIsEr2Fs... 00000040 03 00 00 00 04 00 03 00 02 00 00 00 dc 52 00 00 ............ÜR.. 
Block count: 65638 Free blocks: 6291 Root block: 16514 Journal block: 18 Journal device: 0 Original journal size: 8192 Journal trans. max: 1024 Journal magic: 1460745388 Journal max. batch: 900 Journal max. commit age: 30 Journal max. trans. age: 0 Blocksize: 4096 OID max. size: 972 OID current size: 8 State: 2 (error) Magic String: ReIsEr2Fs Hash function code: 3 Tree height: 4 Bitmap number: 3 Version: 2 Inode generation: 21212
External Links
Wikipedia: ReiserFS
Reports -
Electronic Crime Needs Assessment for State and Local Law Enforcement, National Institute of Justice, March 2002.
NCJ 186276
''Law Enforcement Tools and Technologies for Investigating Cyber Attacks," Institute for Security and Technology Studies, Dartmouth University. June 2002.
National Needs Assessment
Gap Analysis
Strengthening Forensic Science in the United States: A Path Forward, National Research Council, The National Academies Press, 2009.
Strengthening Forensic Science NAP Page
Digital and Multimedia Analysis
Tools memory analysis -
The following tools can be used to conduct memory analysis.
Memory Analysis Frameworks
Volatility Framework - A complete framework for analyzing Windows, Linux and Mac OSX memory images.
WindowsSCOPE Pro, Ultimate - Comprehensive toolkit for the capture and analysis of Windows physical and virtual memory targeting cyber analysis, forensics/incident response, and education. Software and hardware based acquisition with CaptureGUARD PCIe and ExpressCard.
WindowsSCOPE Live live fetch and analysis of Windows computers on a network from Android smartphones and tablets.
Browser Email Memory Tool
pdgmail is a python script to extract gmail artifacts from memory images. Made for images extracted with pdd, but works with any memory image.
Instant Messenger Memory Tool
Belkasoft Evidence Center is a tool by Belkasoft which allows for retrieving various Instant Messenger artifacts from an attached memory image.
Platform Independent Tools
A list of tools which should work regardless of future incremental OS / hardware updates.
inVtero.net Open Source Hypervisor/Process/Kernel detection for Windows, FreeBSD, OpenBSD and NetBSD. inVtero.net is based on interpreting low-level hardware defined constructs which change little over time. See github for details.
Analysis support libraries/tools
Additional tools/libraries that are meant to be used in combination with existing frameworks or tools.
1 Reloc interfaces with a hosted server that has compiled over 200000 fragments of PE relocation data that can be used to transform executable's extracted from memory back into their original state. This means an exact recovery from any dump can be built. This will allow you to validate a secure hash (SHA256, etc...) to have total confidence there exists no backdoor or other malicious patch to a code in memory.
Sanitization standards -
Here are some of the standards by country that we have been able to find regarding the disk sanitization problem:
Australia
ASCI 33: 5 pass wipe, 1 pass with character, 1 pass with inverse of character, repeat first two passes, 1 pass random.
Canada
RCMP TSSIT OPS-II: 8 pass wipe.
Germany
VSItR: Verschlusssachen-IT-Richtlinien, 7 pass wipe followed by verification.
Russia
Gostechcommission management directive, 2 pass with random data.
UK
BHMG Infosec Standard no.5: Three pass wipe followed by verification.
USA
NIST 800-88: Guidelines for Data Sanitation, Sept 2006.
DoD Destruction: Disposition of Unclassified DoD Computer Hard Drives, Assistant Secretary of Defence, June 4, 2001.
DoD 5200.28-STD: Department of Defence Trusted Computer System Evaluation Criteria, December 26, 1985.
DoD 5220.22-M: National Industrial Security Program Operating Manual, January 1995, incorporating Change One (July 1997) and Change Two (February 2001).
NAVSO P-5239-26: US Navy standards for RLL and MFM encoded drives.
Other
Gutmann Wipe: Secure Deletion of Data from Magnetic and Solid-State Memory by Peter Gutmann. Overwrite process using a sequence of 35 consecutive writes. First published in the Sixth USENIX Security Symposium Proceedings, San Jose, Ca, July 22-25, 1996.
Schneier Wipe: Two pass of specific characters followed by five passes of Pseudo Random Data. Published by Bruce Schneier in Applied Cryptography, 1996
Sans institute -
SANS Institute was established in 1989 as a cooperative research and education organization. Its programs now reach more than 165,000 security professionals around the world. A range of individuals from auditors and network administrators, to chief information security officers are sharing the lessons they learn and are jointly finding solutions to the challenges they face. At the heart of SANS are the many security practitioners in varied global organizations from corporations to universities working together to help the entire information security community. SANS is the most trusted and by far the largest source for information security training and security certification in the world. It also develops, maintains, and makes available at no cost, the largest collection of research documents about various aspects of information security, and it operates the Internet's early warning system - the Internet Storm Center.
Sans investigative forensic toolkit workstation -
The SANS SIFT Workstation is a VMware Appliance, built on top of Ubuntu, that is preconfigured with all the necessary tools to perform a forensic examination. It is compatible with Expert Witness Format, Advanced Forensic Format (aff) evidence formats.
Overview
SIFT Workstation is based on Ubuntu.
Software Includes:
The Sleuth Kit
ssdeep & md5deep
Foremost/Scalpel
Wireshark
HexEditor
Vinetto (thumbs.db examination)
Pasco
Rifiuti
Volatility Framework
DFLabs PTK (GUI Front-End for The Sleuth Kit
Autopsy (GUI Front-End for The Sleuth Kit
The SIFT Workstation will allow evidence to be viewed from a Windows workstation. The /images directory and the evidence mount point, the /mnt/hack directory, can be viewed from the local Windows operating system.
Links
Computer Forensics and e-Discovery downloads
Ufs tornado -
The UFS3-Tornado (Universal Flasher Software) is a USB flasher device made by [SarasSoft]. It offers USB data transfer from a phone to a PC.
Features
UFS3 supports phones manufactured by Samsung, SonyEricsson/Ericsson, Motorola-Acer, and Siemens.
UFS operates on Windows 98SE, ME, 2000, and XP.
Piecewise hashing -
Piecewise hashing is a hashing technique to divide an input into chunks and hash each chunk separately. It was invented by Nick Harbour in dcfldd to improve the reliability of disk imaging. The theory was that if one chunk became corrupted, only the integrity of that chunk would be threatened. The piecewise hashes of the other chunks, which hadn't changed, could be used to prove their integrity.
References
md5deep, hashdeep, etc.
dcfldd
See Also
Context Triggered Piecewise Hashing
Portable executable format -
The Portable Executable Format is used by Windows to store executable code. The format is also used to store other types of data, e.g. like Multilingual User Interface (MUI) strings and EventLog messages.
See Also
Windows Event Log (evt)
Windows XML Event Log (evtx)
External Links
Wikipedia: Portable Executable
PE and COFF Specification, by Microsoft
Blog about pefile, by Ero Carrera
Executable (EXE) file format, by the libexe project, October 2011
Tools
pefile is a Python module to read and work with PE (Portable Executable) files, by Ero Carrera
Resources
Resource Hacker, by Angus Johnson
Remnant data -
Remnant data is data that can be recovered from magnetic media after new information has been written to that media. Although remnant data has been recovered from magnetic tapes and floppy disks, there is no credible report that has been published in the open literature that remnant data can be recovered from modern hard drives.
References
Detection of Digital Information From Erased Magnetic Discs, by Venugopal V. Veeravalli , Master's Thesis, Carnegie Mellon University, 1987
See Also
Residual Data
Sector hashing -
Bibliography
Young J., Foster, K., Garfinkel, S., and Fairbanks, K., Distinct sector hashes for target file detection, IEEE Computer, December 2012
Foster, Kristina, "Using Distinct Sectors in Media Sampling and Full Media Analysis to Detect Presence of Documents from a Corpus," Master's Thesis, Naval Postgraduate School, September 2012.
Garfinkel, Simson, Vassil Roussev, Alex Nelson and Douglas White, Using purpose-built functions and block hashes to enable small block and sub-file forensics, DFRWS 2010, Portland, OR (Acceptance rate: 40%, 16/39)
Farrell, P., Garfinkel, S., White, D. Practical Applications of Bloom filters to the NIST RDS and hard drive triage, Annual Computer Security Applications Conference 2008, Anaheim, California, December 2008. (Acceptance rate: 24%, 42/173)
Upcomingevents -
Follow-up Workshop for Digital Forensics Educators
This is a short note to invite you back to the Follow-up Workshop for Digital Forensics Educators at Erie Community College, Tuesday, August 16, 2011 from 9-4. If you hadn’t participated last year…you’re still invited.
Dr. Garfinkel has requested that we have a more interactive workshop with participants sharing their methodologies and course work with the group. This would facilitate more sharing amongst colleagues and give him a better focus regarding the forensics corpora course integration.
We would like to start lining up presenters so that the program can start taking shape. The format for sharing can be as simple as a short talk about your program or PPTs or group activities. Please respond with what you would like to do. I look forward to seeing you in August. Regards, Donna
Registration Link (http://kuroski.net/Forensics2011/login.php)
Setup api logs -
Windows XP:
\Windows\setupapi.log \Windows\setupapi.log.old 
Windows 7 and later:
\Windows\inf\setupapi.app.log \Windows\inf\setupapi.dev.log \Windows\inf\setupapi.offline.log 
Windows 10 and later:
\Windows\inf\setupapi.upgrade.log 
External Links
Understanding Failures and Log Files, by Microsoft
Umpc -
UMPC (Ultra-Mobile-PC) is a specification for a smaller version of tablet PC. Major uses for the device are Internet access and media playback. The device also provides the similar functionality of a regular laptop or desktop PC.
Internet Access
UMPC are able to access internet via WiFi or Ethernet and utilize the Internet to provide access to email, Instant Messaging and VoIP.
Operating System
UMPC uses a standand Microsoft Windows XP Tablet operating system. However, all UMPC's will be capable of running most operating systems.
Hardware
Processor: 900 MHz Intel Celeron M; Intel Pentium M; VIA C7-M RAM: 256 MB - 1 GB Hard disk drive: 30 - 160 GB
Features
Camera
GPS
Bluetooth
Wi-Fi
Ethernet
TV tuner
memory card reader
stereo speakers
fingerprint readers
Programming -
Although this is a wiki largely about digital forensics, understanding how code came to be and the challenges involved can be considered good practice and can help in understanding how an attacker was able to compromise the computer systems of an organization, why the software you're analyzing behaves in a certain way.
External Links
C Programming
What Every C Programmer Should Know About Undefined Behavior #1/3, by Chris Lattner, May 13, 2011
What Every C Programmer Should Know About Undefined Behavior #2/3, by Chris Lattner, May 14, 2011
What Every C Programmer Should Know About Undefined Behavior #3/3, by Chris Lattner, May 21, 2011
Tools
C/C++ Program validation
AddressSanitizer: a fast memory error detector
Skype -
External Links
An Analysis of the Skype Peer-to-Peer Internet Telephony Protocol, by Salman A. Baset and Henning Schulzrinne, September 2004
Guide for Network Administrator’s, by Skype Limited, December 2006
IT Administrators Guide, by Skype Limited, 2010
Skype forensics, by Warren Post, June 21, 2012
Extracting Evidence from Destroyed Skype Logs and Cleared SQLite Databases, by Igor Mikhaylov, November 26, 2013