hemu1438/forensicsgpt-training-data huggingface dataset:
Wetstone technologies inc -
WetStone Technologies sells products and training for computer forensics. These products include:
Gargoyle Investigator
LiveWire Investigator
LiveDiscover
Stego Suite
Their training courses include:
Hacking BootCamp: Exploits and Live Incident Investigation
Advanced Steganography Demystifying Steganography Investigation
Zombies and Botnets: Setup-Investigate-Shutdown
Introduction to Steganography: Steganography and Data Exfiltration
The Art of Keylogging
External Links
Official Website
User account control (uac) -
User Account Control (UAC) is a Windows sub-system introduced in Windows Vista that limits application software to standard user privileges until an administrator authorizes an increase or elevation.
The file virtualization part of UAC is also referred to as LUA (LUAFV.SYS).
EventLogs
Related EventLogs:
C:\Windows\System32\winevt\Logs\Microsoft-Windows-UAC%4Operational.evtx C:\Windows\System32\winevt\Logs\Microsoft-Windows-UAC-FileVirtualization%4Operational.evtx 
External Links
Wikipedia: User Account Control
Vista UAC: The Definitive Guide, by Thomas Hruska, March 21, 2008
You Are Not Admin with UAC, by Corey Harrell, October 8, 2012
UAC Impact on Malware, by Corey Harrell, March 4, 2013
Slack -
Definition
In computer forensics slack refers to the bytes after the logical end of a file and the end of the cluster wherein the final byte of the valid file resides.
File Slack Types
RAM Slack
The data storage space that exists from the end of a file to the end of the last sector assigned to a file.
Drive Slack
The data storage space that exists from the end of the last sector assigned to a file to the end of the last cluster assigned to a file.
Hibernation Slack
Previous hibernation data that has not yet been overwritten in a hibernation file.
See Also
FAT#File_Slack
Usnjrnl -
Parses Windows \$UsnJrnl:\$J files. This crate contains a library and a binary.
!!! Keep in mind that this is not fully tested yet and may contain one or the other bug !!!
Tools
The usnjrnl package contains the following tools:
usnjrnl_dump, which dumps the contents of a \$UsnJrnl:\$J file.
History
usnjrnl was created by Jan Starke in 2021.
See Also
New Technology File System (ntfs)
External Links
Project site
Smart -
SMART is a commercial forensics software package distributed by ASR Data.
Features
File Systems Understood
File Search Facilities
Lists allocated and unallocated files.
Sorts files by type.
Searches for keywords and regex.
Registry viewer.
Historical Reconstruction
Can it build timelines and search by creation date?
Searching Abilities
Can use basic keyword searching.
Hash Databases
SHA-1
MD5
CRC
Evidence Collection Features
"Just about everything you do is logged in SMART. You can selectively export these log events into a simple HTML report."
History
License Notes
SMART is a commercial product with seperate pricing for law enforcement and commercial users.
Pysim -
pySIM (also called SimReader) is a relatively simple open source program that accomplishes the following tasks: - Downloads phonebook entries from a SIM Card - Downloads SMS messages from a SIM Card - Imports .txt files from other acquisitions
Requirements
Windows 98 or later
Smart Card Base Components if using Win98, ME & NT 4.0
GSM SIM Card Reader
Benefits
Cost of Ownership: Since pySIM is freeware it is not cost prohibitive
Ease of use: The simplicity of pySIM enables novice investigators to acquire data
Logging capabilities: Some competing SIM readers do not keep a logfile
Win32.shiz -
Win32.Shiz (other name: Trojan.PWS.Ibank) is a malicious program designed for online banking fraud.
Current versions of this program support almost all online banking systems in Russian Federation (BS-Client, Sberbank, etc) as well as many online poker clients.
The program can be used to steal usernames and passwords used to access online banking system, cryptographic keys stored as files (e. g. on USB Flash drives or hard disk drives), URLs used to access online banking system and certificates stored in web-browsers. If online banking system uses access control lists based on IP addresses then Win32.Shiz can set up a SOCKS proxy-server on a compromised computer.
Python -
Python is a high-level, general-purpose, interpreted, dynamic programming language. Its design philosophy emphasizes code readability, and its syntax allows programmers to express concepts in fewer lines of code than would be possible in languages such as C++ or Java. The language provides constructs intended to enable clear programs on both a small and large scale.
In digital forensics, Python is slowly replacing Perl as the de facto language for short, single purpose written code. Scripts written in Python can often function on both Windows and *nix operating systems with no or minor modifications.
There are several FOSS and commercial implementation of Python interpreters.
Virtual disk image (vdi) -
Image structure
VirtualBox Virtual Disk Image (.vdi) files are structured in essentially the same way:
A header descriptor (512 bytes)
An image block map: If the (maximum) size of the virtual HDD is N MB, then this map is 4N bytes long.
Block alignment padding: The header format allows for padding between the image block map and the image blocks, and (as of version 1.6.2) the CreateVDI function adds padding after the map to ensure that the first image block begins on a 512byte sector boundary. Since the allocation unit on both NTFS and Ext3 file systems is 4096 bytes, you will therefore get slightly better performance (typically a few %) if you make your VDIs (1024N – 128) MByte long.
Up to N x 1MByte image blocks.
See Also
Disk Images
External Links
1, All About VDI's (VirtualBox forums)
Tools
libvhdi
VDI Data Recovery - Data recovery software for VDI (Virtual Disk Image) files.
Spyware -
Spyware is software installed physically on the device with the intent to steal data such as SMS or location data. This data is collected over a period of time. The spyware application then sends this data to the person who installed the spyware to their mobile device or email. Spyware by itself is legal to use. However, it is illegal to install spyware onto a person's device without their consent.
External linkes
Wikipedia: Spyware
Windows update -
TODO: general windows update intro.
Windows Update Status
The following registry keys and values indicate Windows Update status.
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\WindowsUpdate\Auto Update\Results\Detect 
Entry Name Data Type Value 
LastError	REG_DWORD	0 for success	
LastSuccessTime	REG_SZ	UTC time string of the last successful contact with Windows Update servers, e.g. 2013-05-14 12:41:26	
			
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\WindowsUpdate\Auto Update\Results\Download 
Entry Name Data Type Value 
LastError	REG_DWORD	0 for success	
LastSuccessTime	REG_SZ	UTC time string of last successful update download e.g. 2013-05-14 12:41:26	
			
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\WindowsUpdate\Auto Update\Results\Install 
Entry Name Data Type Value 
LastError	REG_DWORD	0 for success	
LastSuccessTime	REG_SZ	UTC time string of last successful update install e.g. 2013-05-14 12:41:26
Qcow image format -
QEMU files to store a disk image.
MIME types
File signature
QCOW files start with hexadecimal: 0x51 0x46 0x49 0xfb (ASCII: "QFI.")
File types
Currently there are three versions of the QCOW1 format; namely QCOW (version 1) and QCOW2 (version 2). Version 3 is considered an extended variant of QCOW2.
Contents
The QCOW format is used to store storage media data, e.g. disk images.
The QCOW image file consists of:
the file header
the L1 table (cluster aligned)
the reference count table (cluster aligned)
reference count blocks
snapshot headers (8-byte aligned on cluster boundary)
clusters containing:
L2 tables
storage media data
The clusters are 512 bytes of size
Encryption
The QCOW2 format supports optional AES encryption
See Also
Disk Images
External Links
The QCOW Image Format, by Mark McLoughlin, June 2006
The QCOW2 Image Format, by Mark McLoughlin, September 2008
QEMU Copy-On-Write file format, by the libqcow project, December 2010
Tools
libqcow
QCOW Data Recovery - Recover data from corrupt or crashed QCOW, QCOW2 virtual image files.
Sql mdf viewer -
SQL MDF Viewer application is a freeware tools that can read and preview contents of SQL MDF files without the requirement of an SQL Server environment.
Features
Entire contents of an SQL MDF file whether corrupt or orphaned, are readable by the software including; triggers, columns, classes, tables, etc., via quick scan option. The scanned database can be saved as an STR for later use.
Note: The software can be used infinitely and doesn’t expire at any particular time duration.
External Links
Official Website
Wireless forensics -
Wireless forensics is the process of capturing information that moves over a wireless network and trying to make sense of it in some kind of forensics capacity.
Wireless Local Area Networks
WLANs are standardized under the IEEE 802.11 series.
Common encryption technologies used by these networks are: WEP, WPA/WPA2-PSK, some networks have no encryption at all.
In order to decrypt intercepted secured WLAN traffic you should crack the encryption key. Note, that the only option for cracking WPA/WPA2-PSK keys is to do a brute-force password guessing attack. There are several WPA-PSK rainbow tables available.
Many commercial network forensics systems can intercept and decrypt WLAN traffic, for example:
Mera Systems NetBeholder Mobile
E-Detective Wireless Detective System
Expert Team 3i Tactical System (3iTS))
As well as some open-source tools:
aircrack-ng
WPA/WPA2-PSK cracking-only solutions with GPU acceleration (15-100 times faster than in CPU-only mode):
ElcomSoft Distributed Password Recovery
Pyrit
Virtualization memory analysis -
Virtualization Memory Analysis can be seen as the analysis of virtual machines from the memory of the host system.
External Links
Wikipedia: Virtualization
Wikipedia: Hypervisor
Hypervisor Memory Forensics, by Mariano Graziano, Andrea Lanzi, and Davide Balzarotti
VM discovery and introspection with Rekall, by Jordi Sánchez López, October 3, 2014
inVtero.net. A tool for physical to virtual & VM identification and extraction., by Shane Macaulay. As well as standard OS memory dumping for Windows, FreeBSD, OpenBSD and NetBSD. Requires no OS layer support, i.e. based on hardware page table isolation only.
Hyper-V
Wikipedia: Hyper-V
Analyzing Hyper-V Saved State files in Volatility, by Wyatt Roersma, October 17, 2013
Hyper-V 2012 and 2012 R2 live virtual machine memory acquisition and analysis, by Wyatt Roersma, April 28, 2014
Wmi -
Windows Management Instrumentation (WMI) is the infrastructure for management data and operations on Windows-based operating systems. It is Microsoft's implementation of Web-Based Enterprise Management (WBEM).
WMI uses the Common Information Model (CIM) to represent classes of information. CIM is developed and maintained by the DMTF (formerly known as the Distributed Management Task Force).
The Managed Object Format (MOF) is the language used to describe Common Information Model (CIM) classes.
The CIM database consists of multiple files (OBJECTS.DATA, MAPPING*.MAP, INDEX.BTR and prior to Windows Vista MAPPING.VER) that are located in the following directories:
"C:\Windows\System32\WBEM\repository\" on Vista+ "C:\Windows\System32\WBEM\repository\FS\" on WinXP/Win2003 
There could also be copies of the CIM database files, in the following locations:
"C:\Windows\System32\WBEM\repository.001\" "C:\Windows\System32\WBEM\repository.001\FS\" "C:\Windows\System32\WBEM\repository.002\" "C:\Windows\System32\WBEM\repository.002\FS\" ... 
External Links
Windows Management Instrumentation, by Microsoft
Common Information Model, by Microsoft
Managed Object Format (MOF), by Microsoft
WMI Command-line Tools, by Microsoft
CIM Schemas, by DMTF (formerly known as the Distributed Management Task Force)
Namespaces
Understanding WMI Namespaces
Configuration Manager WMI namespaces and classes for Configuration Manager reports, by Microsoft
Malware and intrusion analysis
Understanding WMI Malware, by Julius Dizon, Lennard Galang, and Marvin Cruz, July 2010
Abusing Windows Management Instrumentation (WMI) to Build a Persistent, Asyncronous, and Fileless Backdoor, by Matt Graeber, 2015
Subverting Sysmon Application of a Formalized: Security Product Evasion Methodology, by Matt Graeber, Lee Christensen, 2018
WmiEventConsumerClassDerivation.ps1, by Matt Graeber, June 21, 2019
Evidence of File Execution
Do You See What I CCM?, by David Pany, Fred House, December 15, 2016
Secret Archives of Execution Evidence: CCM_RecentlyUsedApps, by James Habben, February 28, 2017
WMI_Forensics, by David Pany, Sept 1, 2017
Xfs -
XFS is a 64-bit journaling file system created by Silicon Graphics, Inc (SGI).
External Links
Wikipedia: XFS
XFS Algorithms & Data Structures - 3rd Edition, May 2018, by Silicon Graphics Inc.
XFS Filesystem Structure
Forensic analysis of SGI IRIX disk volume, July 2016, by R. Carbone
Case Study: Computer Forensics: Fragmented Files Recovery Based on XFS File System, March 24, 2017
XFS (Part 1) – The Superblock, May 21, 2018, by Hal Pomeranz
XFS (Part 2) – Inodes, May 23, 2018, by Hal Pomeranz
XFS (Part 3) – Short Form Directories, May 25, 2018, by Hal Pomeranz
XFS (Part 4) – Block Directories, May 31, 2018, by Hal Pomeranz
XFS (Part 5) – Multi-Block Directories, June 6, 2018, by Hal Pomeranz
Tools
Recovering bad data -
A forensic investigator uses techniques that can recover bad data in order to recover data from a disk drive that is malfunctioning.
One popular program for recovering bad data is SpinRite. This program works by turning off the disk drive's own data recovery technology and repeatedly re-reading each bad sector, with the hope that a good read may eventually happen.
The dd_rescue program can't recover from bad sectors, but it can skip around a disk and recover data that is still mostly good. dd_rescue reads the disk forwards until it finds an error; it then goes to the end of the disk and reads backwards. The aimage advanced disk imager implements a similar algorithm.
Recovering deleted data -
When the user requests to delete a file, most modern operating systems do not erase the actual data. Instead, they merely erase a pointer to the file so that the file does not appear in directory listings. These files can be recovered by simply undeleting the file — that is, restoring the directory entry.
For some file systems, such as FAT, the deleted directory entry itself is easily recovered. In these cases the files can be recovered using an undelete program. In other cases, however, the directory entry is not available — perhaps because it was overwritten. In these cases the only way that the file can be recovered is through the use of File Carving.
Using an undelete function on FAT32 can be very dangerous if any files are fragmented. Deleting the file also removes the FAT entry and so the locations of split fragments can no longer be assumed. Forensically, making any changes to a disk is 'not allowed'.
See Also
Carving
The bat header format -
The Bat! v3
Date: Fri, 7 Feb 2025 15:23:52 +0300 From: Username <username@sendinghost.com> Message-ID: <1066947962.20250207152352@gmail.com> To: Username <username@receivinghost.com> Subject: header test. the bat MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Content-Type: multipart/mixed; boundary="----------069134143049E2085" 
Message-ID format:
Message-ID consists of 4 parts
A random string of numbers
A timestamp of the format YYYYmmDDHHMMSS
The @ symbol
The domain of the sending host
As at March 2025, there is an interesting nuance in Message-ID when used with the mail agent's Postponed Sending feature. When a mail's sending is postponed, the Date header will be updated accordingly, but the timestamp within the Message ID will not.
External Links
Tools
Aid4Mail - convert The Bat! email messages to other formats
The Bat! Viewer & Converter View and convert your The Bat! tbb email messages stored in messages.tbb
The Bat! MailXaminer support The Bat! related file extensions and logs
Zero storage carving -
Zero storage carving, also called inline carving, is the concept of using techniques to enable doing carving of meaningfull and processable chunks or files of uncompressed unencoded data on disks, disk-images or container files without the need for additional storage to be allocated for copies of the relevant data chunks or files. Zero storage carving is sometimes also referred to as in-line carving.
Tools with support or facilities for zero storage carving include:
CarvFs
LibCarvPath
MattockFS
OCFA treegraph API
PhotoRec
scalpel
tsk-cp
References -
Network Applications of Bloom Filters: A Survey, Andrei Broder and Michael Mitzenmacher, Internet Mathematics Vol. 1, No 4: 485--509
Beyond bloom filters: from approximate membership checks to approximate state machines, Bonomi et all, ACM SIGCOMM Computer Communication Review archive, Volume 36 , Issue 4 (October 2006)
Burton H. Bloom, Space/time trade-offs in hash coding with allowable errors, Communications of the ACM, v.13 n.7, p.422-426, July 1970
md5bloom: Forensic ﬁlesystem hashing revisited, Vassil Roussev*, Yixin Chen, Timothy Bourg, Golden G. Richard III, 2006 DFRWS. Published by Elsevier Ltd.
Privacy-Enhanced Searches Using Encrypted Bloom Filters, Bellovin and Cheswick, Columbia Univerisity Technical Report CUCS-034-07, 2004 DIMACS 2004 slides
Aac -
The Advanced Audio Coding (AAC) file format is a lossy digital audio encoding format. Files downloaded from the iTunes Music Store are available in this format.
AAC files usually have a file ending of .m4p and contain lots of metadata. The output of the Unix strings command suggests that at least the following information is contained:
Name of the person who downloaded the file from the iTunes Music Store
Email address of that person (sometimes)
Timestamp of when the file was downloaded
Another timestamp (probably the date when the file was put into the iTunes Music Store?)
The usual metadata of music files (artist, album, cover art etc.)
External Links
Wikipedia: Advanced Audio Coding
Mp3 -
The MPEG-1 Audio Layer 3 format is a commonly-used lossy music format.
Format
MP3 files consist of a well defined header, music data, and optional metadata which may be at the beginning or end of the file, depending on the version of ID3 used.
MP3 files may contain ID3 meta-information, including information about the software used to create the MP3.
Steganography
Using programs such as MP3Stego, MP3 files may be used as a base for steganographic content.
Externals Links
Wikipedia: MP3
Regimented potential incident examination report -
Description
The Regimented Potential Incident Examination Report (RPIER or RAPIER) is script based incident response tool released under the GPL by Intel. It is a modular framework.
RAPIER is a Windows NT based information gathering framework. It was designed to streamline the acquisition of information off of systems in a large scale enterprise network. It was designed with a pretty simple to use GUI so that end-users could be walked through execution of the tool on a system.
Contact: rapier.securitytool@gmail.com
Features
Modular Design - all information acquired is through individual modules
Fully configurable GUI
SHA1 verification checksums
Auto-update functionality
Results can be auto-zipped
Auto-uploaded to central repository
Email Notification when results are received
2 Default Scan Modes – Fast/Slow
Separated output for faster analysis
Pre/Post run changes report
Configuration File approach
Process priority throttling
Information Acquired through RAPIER
Complete list of running processes
Locations of those processes on disk
Ports those processes are using
Checksums for all running processes
Memory dumps for all running processes
All DLLS currently loaded and their checksum
Last Modify/Access/Create times (MAC times) for designated areas
All files that are currently open
Net (start/share/user/file/session)
Output from nbtstat and netstat
All open shares/exports on system
Current routing tables
List of all network connections
Layer3 traffic samples
Logged in users
System Startup Commands
MAC address
List of installed services
Local account and policy information
Current patches installed on system
Current AV versions
Files with alternate data streams (ADS)
Files marked as hidden
List of all installed software on system (known to registry)
System logs
AV logs
Copies of application caches (temporary internet files) – IE, FF, Opera
Export entire registry
Search/retrieve files based on search criteria.
External Links
Official website
Google Discussion Group
Presentation at FIRST Conference 2006
Libexif -
libexif is an open source library for reading and editing Exif metadata, commonly found in digital pictures. It is written in C and released under the LGPL. The library runs under GNU/Linux, BSD, MacOS, and Windows. The library is developed by the "libexif project." The project also includes command-line and graphical front-ends.
Examples
The use of the command-line front end is typical of many Unix or Linux commands.
exif example.jpg
This command will write all tags and values contained in the file to standard output. It will also indicate if the file contains an embedded thumbnail.
Other common options for the exif command:
-e extracts the thumbnail. You can use the --output option to a specified file
-r removed the thumbnail from the image. Will write a new image. You can specify the file with the --output option
-l lists all known tags
--remove removes a specified tag or entire IFD (Image File Directory) is no tag is given
External Links
libexif project documentation
man page for exif command line tool
External links -
A Probabilistic Approach to Source Code Authorship Identification Kothari, Jay; Shevertalov, Maxim; Stehle, Edward; Mancoridis, Spiros Information Technology, 2007.
Source Code Author Identification Based on N-gram Author Profiles, Georgia Frantzeskou, Efstathios Stamatatos , Stefanos Gritzalis and Sokratis Katsikas, IFIP International Federation for Information Processing, 2006
Named entity recognition -
Named Entity Recognition (NER) is the process of identifying and extracted named entities such as People, Places, Things and Time from natural language documents.
Named Entity Recognition is a growing part of computer forensics. Please see Wikipedia's article on Named entity recognition for complete information. The remainder of this article stresses aspects of NER that are relevant to current forensics practice and research.
Toolkits
Bibliography
Semi-Supervised Named Entity Recognition: Learning to Recognize 100 Entity Types with Little Supervision, David Nadeau, PhD Thesis, Ottawa-Carleton Institute for Computer Science, School of Information Technology and Engineering, University of Ottawa, 2007
See Also
Wikipedia's article on Named entity recognition
US Patent 7299180 - Name entity extraction using language models
NYU's Name/Entity Extraction Engine
DARPA's Global Autonomous Language Exploitation (GALE) program
GATE - A General Architecture for Text Engineering
Information Extraction with GATE (slides)
Digital forensics phd theses -
@PhDThesis{Kessler2010,
title="Judges’ Awareness, Understanding, and Application of Digital Evidence", author="Gary Craig Kessler", year=2010, institution="Graduate School of Computer and Information Sciences Nova Southeastern University",
} 
@PhDThesis{Carrier2006,
title="A Hypothesis-Based Approach to Digital Forensic Investigations", author="Brian D. Carrier", year=2006, institution="Purdue University",
}