hemu1438/forensicsgpt-training-data huggingface dataset:
Google desktop search -
Google Desktop Search is an application for both Windows and Mac OS X that allows the user to index and then search their local hard drives. These applications integrate with other Google tools such as searching the web and Gmail.
By default, the Mac version caches content that can be recovered even after the original has been deleted.
Google Desktop will be discontinued as of September 14 2011 1
See Also
Windows Desktop Search
External Links
Official website
Wikipedia entry on Google Desktop
Wikipedia list of Desktop search engines
Solveit -
SOLVE-IT (Systematic Objective-based Listing of Various Established (Digital) Investigation Techniques) is a knowledge base for and by the digital forensic community, supported by DFRWS.org, that describes and indexes techniques available to digital forensic investigators during an examination.
Uniquely, it also describes potential weakness at each stage of a digital forensic investigation, including in digital forensic tools. It also provides Python tooling to compile the contents of the knowledge base into different formats, making it useful for a number of different applications.
Use
The many potential applications of SOLVE-IT include:
strengthen tool testing by scoping error-focused data sets for a technique
reinforce techniques by cataloguing available mitigations for weaknesses
bolster quality assurance by identifying potential weaknesses in a specific processes
structure consideration of potential uses of AI in digital forensics
augment automation by highlighting relevant CASE ontology classes
prioritize innovation by identifying academic research opportunities
standardizing language and terminology in teaching using the indexed techniques
conducting capability assessments of labs or individuals against the techniques
One of the most interesting and immediate applications of SOLVE-IT is to avoid missed or unmitigated errors in digital forensic processes. This can be undertaken to review either standard processes, tool workflows, or even individual investigations.
Reference
SOLVE-IT: A proposed digital forensic knowledge base inspired by MITRE ATT&CK from the Proceedings of the DFRWS EU 2025 Conference
External Links
SOLVE-IT GitHub repositories
See Also
ASTM E3016-18: Standard Guide for Establishing Confidence in Digital and Multimedia Evidence Forensic Results by Error Mitigation Analysis based on SWGDE document
Libyal -
The libyal project contains an overview of various projects by the author. It also contains some scripts for maintaining these various projects.
Tools
The libyal project contains the following scripts:
msvscpp-convert.py, which is a script to convert Visual Studio (express) solution and project files from one version to another.
Deprecated tools:
libyal-build.py, which is a script to automate creating builds of libyal libraries. The functionality of this script has been integrated into the l2tdevtools projects.
History
Libyal was created by Joachim Metz in 2012.
External Links
Project site
History of the libyal projects and related publications
Outlook express database (dbx) -
Outlook Express uses the Outlook Express Database (DBX) database to store emails, folders, etc. The DBX was introduced in version 5. Earlier versions of Outlook Express use different file formats.
MIME types
File signature
OE5 DBX files start with hexadecimal: 0xcf 0xad 0x12 0xfe. For now it is assumed that this is the file signature. It is followed by a content class identifier (CLSID), which is unique for the type of DBX file.
File types
Although DBX probably contains a minor/major version (offset: 20 and 24) until now it is unknown how it affects the file type.
Contents
The contents of a DBX file is dependent on the content CLSID.
Message Database (CLSID: 6F74FDC5-E366-11d1-9A4E-00C04FA309D4)
Folder Database (CLSID: 6F74FDC6-E366-11d1-9A4E-00C04FA309D4)
Encryption
External Links
Wikipedia: Outlook Express Wikipedia
libdbx, see FILE-FORMAT in the package.
Resilient file system (refs) -
The Resilient File System (ReFS) (AKA protogon) was introduced in Windows 8, with the intention of eventually replacing NTFS. It has been designed for use with huge data sets, and provides improvements over NTFS with features such as repairing errors "on-the-go" (automatic chkdsk) and preventing Torn Writes by using a "Copy-On-Write" feature.
External Links
Resilient File System overview
Resilient File System (ReFS), by the libfsrefs project, May 2012
Forensic Investigation of Microsoft's Resilient File System (ReFS), by Andrew Head, April 2015
Paladin forensic suite live boot ubuntu -
PALADIN is a completely free forensic investigation tool kit, developed by SUMURI.
Product Overview
PALADIN is a modified “live” Linux distribution based on Ubuntu that simplifies various forensics tasks in a forensically sound manner via the PALADIN Toolbox. PALADIN is available in 64-bit and 32-bit versions.
Product Features
PALADIN Features over 100 open source forensic tools, including:
Anti-virus tools
Carving tools
Database tools
Development tools
Encryption tools
File differential tools
File system tools
Forensic suite
Hardware analysis
Hashing tools
Hex editor
Imaging tools
Internet analysis tools
Memory analysis
Log analysis
Mail analysis
Messenger analysis
Metadata analysis
Mobile device forensics tools
Network analysis
Password discovery
Photo analysis
PLIST Analysis
Recycle bin analysis
Reporting tools
Social media analysis
Stenography tools
Thumbnail analysis
Timeline analysis
TSK
Virtual machines
Windows registry tools
Remote Services Mode
Every version of PALADIN includes SUMURI Remote Service Mode which can be activated at boot. In Remote Services Mode, SUMURI experts can perform a variety of services remotely anywhere in the world.
External Links
Official Website
Cell phone forensics -
Guidelines
If on, switch it off. If off, leave off.
#* Note only under exceptional circumstances should the handset be left switched on and in any case every precaution to prevent the handset connecting with the Communication Service Provider should be made. Consider use of one of many wireless preservation or RF isolation techniques. Note that the slightest signal leakage will allow an overwriting text message through even if a phone call can't get through.
#* Instead of switching off, it may be better to remove the battery. Phones run a different part of their program when they are turned off. You may wish to avoid having this part of the program run.
#* Note that removing the battery or powering off a mobile phone may introduce a handset unlock code upon powering the device on.
Collect and preserve other surrounding and related devices. Be especially careful to collect the power charger. The phone's battery will only last a certain amount of time. When it dies, much of the data on the device may go too!
Plug the phone in, preferably in the evidence room, as soon as possible.
Retain search warrant (if necessary - Law Enforcement).
Return device to forensic lab if able.
Use forensically sound tools for processing. However, also remember ACPO Principle 2 says: In exceptional circumstances, where a person finds it necessary to access original data held on a computer or on storage media, that person must be competent to do so and be able to give evidence explaining the relevance and the implications of their actions.
Notes
Expand on as to what to collect:
ESN
IMEI
Carrier
Manufacturer
Model Number
Color
Other information related to Cell Phone and SIM Card
Process:
Photograph the Cell Phone screen during power up.
Research the Cell Phone for technical specifications.
Research the Cell Phone for forensic information.
Based on phone type GSM, CDMA, iDEN, or Pay As You Go determine acquisition tools
GSM:
Phone and SIM Card
SIM Card
CDMA:
Phone
iDEN:
Three major tools exist for iDEN Phones:
iDEN Companion Pro
iDEN Media Downloader
iDEN Phonebook Manager
Pay As You Go:
Phone
External Links
Articles and Reference Materials
Law Enforcement, Forensics and Mobile Communications
Investigative Support
Mobile-Forensics.com (Research Forum for Mobile Device Forensics)
TREW Mobile Telephone Evidence (Mobile Telephone Evidence Practitioner Site)
Phone Research
GSMArena.com (Technical information regarding GSM Cell Phones)
MobileForensicsCentral.com (Information regarding Cell Phone Forensic Applications)
Training
SANS: FOR585: Smartphone Forensic Analysis In-Depth
Teel Technologies Smartphone training
Mobile-Forensics.com (Research Forum for Mobile Device Forensics)
Paraben-Forensics.com (Paraben's Handheld Forensic Training Classes)
Micro Systemation Training (Mobile Forensics Training)
Property list (plist) -
The Property List (plist) format is widely used within Mac OS X. It is used to store various kinds of information, e.g. configuration.
There are multiple format variations of plists:
ASCII plist
Binary plist
XML plist
JSON plist
External Links
Wikipedia article on Property list
CFBinaryPList.c by Apple
Old-Style ASCII Property Lists
Python Module for dealing with Binary Property Lists (bplists) Documentation
Tools
binplist
Python Module for dealing with Binary Property Lists (bplists)
libplist
munki - Managed software installation for OS X comes with code to handle binary plists
plutil.pl
Property List Editor on Mac OS X
Rich text format (rtf) -
The Rich Text Format (RTF) is a format for text and graphics interchange. RTF uses the American National Standards Institute (ANSI), PC-8, Macintosh, or IBM PC character set to control the representation and formatting of a document.
External Links
Word 2000: Rich Text Format (RTF) Specification, version 1.6
Malware
Between an RTF and OLE2 place: an analysis of CVE-2012-0158 samples, by Paul Baccas, October 2013
Tools
rtfobj - a Python tool to extract embedded objects from RTF files
Spotlight database file format -
The Spotlight database file format is used by Mac OS X to store Spotlight Desktop search information.
MacOS command line tools
MacOS has several command line tools to query a Spotlight database
mdfind - finds files matching a given query
External Links
OSX Spotlight Queries, by Sarah Edwards, December 2015
Shining a light on Spotlight: Leveraging Apple's desktop search utility to recover deleted file metadata on macOS, by Tajvinder Singh Atwal, Mark Scanlon, Nhien-An Le-Khac, October 2018
The user spotlight database, by Yogesh Khatri, October 16, 2018
Tools
spotlight_parser
Wireshark -
Wireshark is a popular network protocol analyzer.
Overview
Wireshark has a rich feature set which includes the following:
Deep inspection of hundreds of protocols;
Live capture and offline analysis;
Standard three-pane packet browser;
Multi-platform: runs on Windows, Linux, Mac OS X, Solaris, FreeBSD, NetBSD, and many others;
Captured network data can be browsed via a GUI, or via the TTY-mode TShark utility;
Powerful display filters;
Rich VoIP analysis;
Read/write many different capture file formats: tcpdump (libpcap), Catapult DCT2000, Cisco Secure IDS iplog, Microsoft Network Monitor, Network General Sniffer® (compressed and uncompressed), Sniffer® Pro, and NetXray®, Network Instruments Observer, Novell LANalyzer, RADCOM WAN/LAN Analyzer, Shomiti/Finisar Surveyor, Tektronix K12xx, Visual Networks Visual UpTime, WildPackets EtherPeek/TokenPeek/AiroPeek, and many others;
Capture files compressed with gzip can be decompressed on the fly;
Live data can be read from Ethernet, IEEE 802.11, PPP/HDLC, ATM, Bluetooth, USB, Token Ring, Frame Relay, FDDI, and others (depending on your platfrom);
Decryption support for many protocols, including IPsec, ISAKMP, Kerberos, SNMPv3, SSL/TLS, WEP, and WPA/WPA2;
Coloring rules can be applied to the packet list for quick, intuitive analysis;
Output can be exported to XML, PostScript®, CSV, or plain text.
Network Forensics
Wireshark can be used in the network forensics process. There are some limitations:
Wireshark is packet-centric (not data-centric);
Wireshark doesn't work well with large network capture files (you can turn all packet coloring rules off to increase performance).
Wireless Forensics
Wireshark can decrypt IEEE 802.11 WLAN data with user specified encryption keys.
Usage
The following YouTube video provides a brief overview of how to use Wireshark to capture and analyze network-based evidence:
https://www.youtube.com/watch?v=bbT0zDIfjWw
Another helpful utility in Wireshark is "Follow TCP Stream", this will bring up a new window that will show all the communications between the server and the client. To get to this feature left click on a packet and hover over "follow", then click on "TCP STREAM".
External Links
Wireshark Wiki
See Also
tcpdump
Text file (txt) -
The Text file (TXT) format consist of 8-, 16- or 32-bit characters that use printable characters along with some control data such as tabs and line feeds. 1 Text files are split into several major types:
DOS/Windows format ends each line using Carriage Return (CR) or char(13) and a Line Feed (LF) or char(10) byte sequence,
Unix format includes only the Carriage Return (CR) or char (13) at the end of the line.
Macintosh format includes only the Line Feed (LF) or char(10) at the end of the line.
Unicode includes an optional encoding in the first two bytes Byte Order Mark (BOM) that identifies the Unicode encoding. This is used to identify little endian or big endian byte order. Unicode defines an 8-bit encoding UTF-8, a 16-bit encoding UTF-16 and a 32-bit encoding UTF-32. Earlier equivalent encodings are respectively UCS-1, UCS-2 and USC-4
EBCIDIC used char(15) for a new line. 2
They are usually ASCII encoded, although other encodings are possible to allow various language scripts to be used. Other encodings include EBCIDIC from the old IBM mainframe. Text files can have the MIME type "text/plain", often with suffixes indicating an encoding (e.g. "text/plain;charset=UTF-8".) Any basic text reader can be used to view the contents of a simple text file, however some (notably Notepad) have issues with certain less popular encodings. Wordpad is included with Windows and may display the files properly.
Translation of a DOS/Windows text file to Unix is performed by removing the Carriage Return from the end of the line. The reverse is simply the addition of the Carriage Return to the Line Feed. Files that have double spaces between the lines may have been improperly translated from one system to another.
Text files usually have the .txt extension. A number of file formats is actually "text files", but bear different extensions. For example is web documents (html text files but is written with a specific syntax so the applications the files are designed to work with can read i correctly. Other kinds of files that can be seen as text files are source code files, XML, etc.
Matriux -
A Linux Live CD intended for penetration testing.
Matriux is a fully featured security distribution, based on Ubuntu. It consists of powerful, free and open source computer forensics and data recovery tools which can be used for forensics analysis and investigation purposes.
Apart from forensics usages Matriux also provides a wide platform to let security professionals utilize the power of open source to perform day-to-day web application penetration testing and server hardening tasks.
Matriux is lite and designed to run from a CD / DVD or USB stick or it can easily be installed to your hard disk in a few steps in no time.
History
An Indian infosec professional Manu Zacharia had initialized the idea of developing Matriux as a security distribution in 2008 and by the end of 2009 on 5th December released the Matriux Lithium version at ClubHack2009 Its successor Matriux Xenon was released on December 4, 2010 at ClubHack2010.
Matriux is also Asia's first OS for hacking, penetration testing, and cyber forensic investigation.
Matriux Krypton, We have officially released it
Tools
Matriux offers many forensics tools under its Arsenal in Digital-Forensics category
Acquisition
Automated Image & Restore 1.28
Guymager
Analysis
Start Autopsy
Autopsy Forensics Browser
Gpart
Pasco
Vinetto
Start WarVOX
Open WarVOX Web Interface
Xplico Console Mode (Internet Traffic Decoder)
Xplico Web Interface (Internet Traffic Decoder)
External Links
Official website
Download Matriux OS
Matriux sourceforge
Artworks
IRC:
Xbox -
History
Xbox is a sixth generation era video game console produced by Microsoft, first released on November 15, 2001 in North America, then released on February 22, 2002 in Japan, and on March 14, 2002 in Europe. The Xbox was Microsoft's first independent venture into the video game console arena, after having developed the operating system and development tools for the MSX, and having collaborated with Sega in porting Windows CE to the Sega Dreamcast console.
What can a mod do?
Mods can vary depending on the system you have. A mdo is basically the machine modified or hacked, allowing the user to do more than they could have prior. Basically, the Xbox can do the most with a mod because of its internal hard drive. By modding the hard drive, it can run games without the original disc, install old system ROM's for playing games from previous generations of gaming systems, and using the Xbox as a file server for various file formats (movies, pictures, audio, etc...) Typically, people boot the Xbox with a Linux kernel, which even allows the user to surf the web. Xbox live can also be ran using a different service, which of course is free.
Why is this Forensically important?
With the Xbox able to do all of the above, this leaves criminals endless capabilities. Who would ever think to look at an Xbox for evidence? Everything someone would extract from a computer, can now be found on a gaming system like the Xbox. The mod allows one to transform their Xbox to a file server, which can include incriminating evidence. Learning how to extract information from the Xbox is a very important detail that shouldn't be overlooked.
External Links
Wikipedia: Xbox
Bibliography
XFT: a forensic toolkit for the original Xbox game console, by David Collins
Vmware virtual disk format (vmdk) -
Image types
There are multiple types of VMWare Virtual Disk Format (VMDK) files:
2GbMaxExtentFlat (or twoGbMaxExtentFlat); descriptor file (name.vmdk) with RAW extent data files (name-f###.vmdk). This image type is basically a split RAW image.
2GbMaxExtentSparse (or twoGbMaxExtentSparse); descriptor file (name.vmdk) with VMDK sparse extent data files (name-s###.vmdk)
monolithicFlat; descriptor file (name.vmdk) with RAW extent data file (name-f001.vmdk). This image type is basically a RAW image.
monolithicSparse; VMDK sparse extent data file (name.vmdk) which contains the descriptor file data.
Descriptor file
The descriptor file defines how and where the data of the VMDK image is stored. The data is stored in extent data files.
Extent data file types
There are multiple types extent data files:
RAW extent data file or device
VMDK sparse extent data file
COWD sparse extent data file
Note that the VMDK sparse extent data file can be in a dirty state (or uncleanShutdown).
See Also
Disk Images
External Links
Virtual Disk Format 5.0, by VMWare
VMware Virtual Disk (VMDK) format specification, by the libvmdk project
Tools
libvmdk
VMWare VMDK Data Recovery - Allows you to recover data from corrupt and crashed VMDK VMware virtual machine disk image files.
The Technology -
Hard drive manufacturers have moved to a new standard: 4KB (4,096 bytes) sectors, replacing 512B sectors. This is a good thing; it means that the signal-to-noise ratio improves, and less space is needed for error correction. Long-term improvements in speed, density, and overall capacity. Western Digital has started releasing drives with 4KB sectors under the name "Advanced Format" (not to be confused with the Advanced Forensics Format.
The Standard
ATA 7 (T13/D1532, INCITS 397-2005) introduced Long Physical Sector (LPS) and Long Logical Sector (LLS) feature sets. Drives with large sector sizes shall report the actual physical/logical size in words 106 and 117-118 of the ATA IDENTIFY data.
Some Western Digital drives with "Advanced Format" reportedly do not provide the information about physical sector size.
The Problem: Death of LBA 63
Operating systems written before the transition, particularly XP, have trouble with the new drives. XP makes an assumption about where the format should start (LBA 63), but this doesn't work well with the translation software that maps from logical 512B blocks to physical 4K blocks.
The nutshell is that XP should not be used to format these drives, and some assumptions made by tools and users need to be corrected. For analysis purposes, note that you can't assume that an NTFS partition starts at LBA 63. If you are used to using, for example, the Sleuthkit command "fls -o 63 ", this may need to change. Hopefully more information about these drives will come forth as time progresses.
LBA 63 was dying anyway. Windows Vista and Windows 7 both align to LBA 2048 by default. This change happened before the Advanced Format 512e drives hit the marketplace.
The Solution
To format one of these drives properly for Windows XP, use the following utility (this applies only to drives from Western Digital):
Western Digital Advanced Drive Format Utility
External Links
A Good Overview at AnandTech
Advanced Format Technology - White Paper
A Tom's Hardware Review of the WD Caviar Green Drives
ATA 4 KiB sector issues
4K-sector drives and Linux (LWN.net)
WD6400AARS-00Y5B1 does not provide sector size info (smartmontools ticket)
Webloc -
This is Mac OS X's internet shortcut file, similar to the Microsoft Windows .URL file. However, due to the heavy usage of AppleDouble resources, the expected data is actually stored as metadata and not in the the expected data file location.
On the file system itself, the shortcut file named News.webloc pointing to http://news.google.com is actually a zero byte file. The URL is instead stored in the ._News.webloc file in a field with resource type 'url'.
The contents of this resource file are visible with the Apple Developer Tool DeRez
Forensics of virtualization products -
Dealing with Virtual Machine Images
It is becoming increasingly common to find evidence drives with Virtual Machines (VM) on them. The VMs may contain evidence of their own, but with their unique file structure, care must be taken during examination. Running the virtual machine could destroy artifacts that are important.
Virtual Box
There are two methods for creating a way to mount or inspect a Virtual Box VM. Virtual Box disks typically have the extension "vdi" for Virtual Desktop Infrastructure. The mount method requires a Linux system and "qemu". The other method converts a vdi to a raw image format which can then be inspected with traditional forensics tools.
Mount
Install qemu-kvm using your preferred installation tool (apt-get, etc)
Load the network block device module >sudo modprobe nbd
Use Qemu to load the VDI file as a loop back device >sudo qemu-nbd -c /dev/nbd0 infile.vdi
Mount >sudo mount /dev/nbd0p1 /mnt
Inspect the file system as needed
To undo:
>sudo umount /mnt
>qemu-nbd -d /dev/nbd0
Convert
Conversion requires the Virtual Box tool kit, if you don't already have it.
Install virtualbox-ose using your preferred installation tool (apt-get, download from VirtualBox.org, etc)
Convert to raw format >VBoxManage internalcommands converttoraw infile.vdi outfile.img
Inspect the raw image as per usual, either with TSK, EnCase, or mount