hemu1438/forensicsgpt-training-data huggingface dataset:
FreeBSD -
This section describes how to set up a FreeBSD system as a disk imaging system.
Install FreeBSD 6.2 on a new computer
Boot the FreeBSD 6.2 CDROM
Hit return to boot the Default
Hit return to select "United States" (or choose your country)
Hit down-arrow and hit return to select "standard install"
Setting up the partition table:
Enter to select "OK." If the geometry is incorrect, enter "OK" to accept.
If there are any partitions, use the up and down arrows to select them and press "d" to delete them.
press "a" to use All of the disk.
press "q" to finish
press the down arrow and hit Enter to select the Standard MBR (no boot manager)
Setting up the FreeBSD partitions:
Press "enter" at the OK prompt.
Press "a" for auto-defaults
Press "q" to accept
Choosing what to install:
Press the down arrow and Enter to select "all" software
Press Enter at the "yes"
Press the up arrow and press Enter to Exit
Press Enter to select CD/DVD
Press Enter to confirm
FreeBSD 6.2 will be installed. Now you need to configure it.
Press Enter at the OK prompt when installation is complete.
Press [Yes] Enter to configure an Ethernet address.
Press [Ok] Enter to configure the first ethernet card.
Press [No] Enter when asked if you want to configure an IPv6 interface.
Press [Yes] Enter when asked if you want to configure with DHCP.
Press [No] Enter when asked if you want to be a network gateway.
Press [No] Enter when asked if you want to configure inetd.
Press [No] Enter when asked if you want to enable SSH login.
Press [No] Enter when asked if you want to have anonymous FTP.
Press [No] Enter when asked if you want to configure the machine as an NFS server.
Press [No] Enter when asked if you want to configure the machine as an NFS client.
Press [No] Enter when asked if you want to customize the system console settings.
Press [Yes] Enter when asked if you want to set the machine's time zone.
Press [No] Enter when asked if the machine's system clock is in UTC.
Select your region and press [OK] Enter
Select your country and press [OK] Enter
If you are in the US, Select your time zone and press [OK] Enter
Press [Yes] Enter to confirm the time zone.
Select [No] Enter when asked if you need Linux compatibility.
Select [No] Enter when asked if you have a mouse (even if you have one).
Select [No] Enter when asked if you wish to browse the ports collection.
Select [No] Enter when asked if you wish to add any user accounts.
Press [OK] Enter when told you will be setting the Root password
Press [Enter] for the Root password; we will use no password.
Press [Enter] to confirm the empty root password.
Getting Your Forensics Software Working for local analysis
Note that the order you do this matters: Sleuth Kit won't compile with AFFLIB support unless AFFLIB is installed on your system.
Download and install libewf if you want EnCase support.
Download and install [AFFLIB](aff.md
Download and install The Sleuth Kit from http://www.sleuthkit.org/
Download and install fiwalk
FreeBSD -
To mount a disk image on FreeBSD:
First attach the image to unit #1:
# mdconfig -a -t vnode -f /big3/project/images/img/67.img -u 1
Then mount:
# mount -t msdos /dev/md1s1 /mnt
# ls /mnt BOOTLOG.PRV BOOTLOG.TXT COMMAND.COM IO.SYS MSDOS.SYS
To unmount:
# umount /mnt # mdconfig -d -u 1
To mount the image read-only, use:
# mdconfig -o readonly -a -t vnode -f /big3/project/images/img/67.img -u 1 # mount -o ro -t msdos /dev/md1s1 /mnt
Linux
To mount a disk image on Linux
# mount -t vfat -o loop,ro,noexec img.dd /mnt
The ro is for read-only.
This will mount NSRL ISOs:
# mount /home/simsong/RDS_218_A.iso /mnt/nsrl -t iso9660 -o loop,ro,noexec
Some raw images contains multiple partitions (e.g. full HD image). In this case, it's necessary to specify a starting offset for each partition.
# mount -t vfat -o loop,offset=32256,ro,noexec img.dd /mnt/tmp_1 # mount -t vfat -o loop,offset=20974464000,ro,noexec img.dd /mnt/tmp_2
kpartx
Mounting raw images with multiple partitions is easy with kpartx. Type aptitude install kpartx as root to install kpartx under Debian. kpartx is creating device-mappings for each partition. If the raw image looks like this:
Device Boot Start End Blocks Id System rawimage.dd1 1 1 8001 83 Linux rawimage.dd2 2 2 8032+ 5 Extended rawimage.dd5 2 2 8001 83 Linux
The command
# kpartx -v -a rawimage.dd
creates these mappings
/dev/mapper/loop0p1 /dev/mapper/loop0p2 /dev/mapper/loop0p5
The partitions can be mounted with these commands:
# mount /dev/mapper/loop0p1 /media/suspectHD_01/ -o ro # mount /dev/mapper/loop0p5 /media/suspectHD_02/ -o ro
Don't forget the switch -o ro !
To unmount
# umount /mnt
Windows
MS Windows does not include a native means for mounting acquired images. However, there are tools available for mounting acquired images on Windows systems.
Free Tools
Arsenal Image Mounter - Arsenal Image Mounter takes the contents of disk images and presents them to Windows as SCSI disks
FTK Imager
ImDisk Virtual Disk Driver
Paraben P2X
Commercial Tools
SmartMount
Mount Image Pro - has a 14-day trial version
Early userspace -
According to the Linux documentation, "early userspace" is a set of libraries and programs that provide various pieces of functionality that are important enough to be available while a Linux kernel is coming up, but that don't need to be run inside the kernel itself.
Early userspace is commonly used in various Linux Live CDs and Live USBs for loading necessary device and file system drivers, as well as for other preparatory actions (e.g. setting up a framebuffer).
Early userspace is also responsible for mounting a root file system, which contains a live system (desktop environment and various applications reside in a live system, while early userspace contains only a limited set of programs required for booting up).
Due to varied conditions in which Live CDs and Live USBs are booting up (for example, it is possible to make Live USB from Live CD by writing ISO 9660 image directly to USB device as well as by exporting files from ISO 9660 image to an existing file system on USB device and setting up a bootloader on this device), early userspace should locate a root file system first. A root file system can be stored in a SquashFS image file, a raw image file, a partition with a file system, a device without a partition table (but with a file system), or even in a set of directories in unpacked form (although specific implementations of early userspace may not support everything listed above).
After booting, contents of a root file system are visible as the contents of "/" directory.
How to analyse partitions -
A How-to for dealing with partitions.
Sleuth Kit Informer #12 suggests using the mmls program to display the contents of partitions.
For example:
# mmls -t dos disk.dd Slot Start End Length Description 00: ----- 0000000000 0000000000 0000000001 Primary Table (#0) 01: ----- 0000000001 0000000062 0000000062 Unallocated 02: 00:00 0000000063 0002056319 0002056257 Win95 FAT32 (0x0B) 03: 00:01 0002056320 0008209214 0006152895 OpenBSD (0xA6) 04: 00:02 0008209215 0019999727 0011790513 FreeBSD (0xA5)
You can use mmls to examine the OpenBSD and FreeBSD partitions that are inside the DOS partition:
# mmls -t bsd -o 2056321 disk.dd Length Description 00: 02 0000000000 0019999727 0019999728 Unused (0x00) 01: 08 0000000063 0002056319 0002056257 MSDOS (0x08) 02: 00 0002056320 0002260943 0000204624 4.2BSD (0x07) 03: 01 0002260944 0002875823 0000614880 Swap (0x01) 04: 03 0002875824 0003080447 0000204624 4.2BSD (0x07) 05: 04 0003080448 0003233663 0000153216 4.2BSD (0x07) 06: 07 0003233664 0004257791 0001024128 4.2BSD (0x07) 07: 06 0004257792 0008209214 0003951423 4.2BSD (0x07) 08: 09 0008209215 0019984859 0011775645 Unknown (0x0A)
(Examples from Sleuth Kit Informer #12)
External Links
Sleuth Kit Informer #12: Using mmls from The Sleuth Kit
How to image an ide disk with aimage and freebsd -
Here is a photo of my disk imaging system:
Key elements of the disk imaging system:
You need to have an internal IDE card which is not used for anything but disk imaging;
You need to have an external hard drive power supply, so that you can power the IDE drives without using your computer's power supply (if you use your computer's power supply, you can easily crash your computer when attaching or detaching the power supply);
Imaging Checklist
Set up a disk imaging station;
You should have a 50-pin IDE ribbon cable going from your IDE controller to the desktop;
Do not connect your imaging drive yet!
Boot the computer into FreeBSD;
Attach the IDE hard drive to the ribbon cable FIRST;
Now, attach power to the IDE drive;
You need to determine which ATA port the IDE drive is now connected to. In all likelihood it is ata0, ata1, ata2 or ata3. If you have an internal hard drive on an IDE interface, then the internal interface is probably ata0 and ata1 and the external is probably on ata2 or ata3;
You also need a place to store the AFF files you are going to be creating. I usually put them in /usr/affs which is a directory you will need to create;
Log in as root;
mkdir /usr/affs
Now, try to image the drive with this command:
aimage ata2 /usr/affs/disk1.aff
If this doesn't work, try:
aimage ata3 /usr/affs/disk1.aff
If it works, you'll see the aimage program running.
What can go wrong
aimage may not be installed. If you get the error message "aimage: command not found" then you need to install AFFLIB and then make sure that the aimage command (usually installed in /usr/local/bin) is in your PATH. You can check this out by running /usr/local/bin/aimage instead of aimage;
Your source drive can be broken, aimage should tell you this;
You can run out of disk space. You need a LOT of disk space to store disk images — figure 30GB to image a 60GB drive.
What to do after you have made your images
Once you have made a few images, you'll need to put them somewhere. Typically this means uploading them to a server.
See Also
How To Set Up a Disk Imaging Station
How To Ship Drives
Windows nt registry file (regf) -
Microsoft Windows NT 4 (and later) uses the Windows NT Registry File (REGF) to store system and application related data, e.g. configurations, most recently used (MRU) files.
MIME types
File signature
REGF has the following file signature:
hexadecimal: 72 65 67 66
ASCII: regf
File types
There are multiple types of REGF files:
primary file (other names: normal file, data file)
transaction log file
external file
backup file
Primary files, external files (created with the RegSaveKey() routine), and backup files share exactly the same format.
Transactional Registry (TxR)
In Vista the Transactional Registry (TxR) was introduced. TxR creates transaction log files similar to:
%FILE%{%GUID%}.TM.blf
%FILE%{%GUID%}.TMContainer00000000000000000001.regtrans-ms
%FILE%{%GUID%}.TMContainer00000000000000000002.regtrans-ms
Where %FILE% is the name of the REGF normal (data) file, e.g. NTUSER.DAT and %GUID% a string representation of a GUID/UUID.
TxR is similar to Transactional NTFS (txf) and uses the Common Log File System (clfs).
Contents
The REGF basically consists of a set of hive bins. These hive bins contain cells that make up a hierarchy of keys and values.
Artifacts
The on-disk format itself contains several artifacts:
last written timestamp of a registry key;
access bits of a registry key (starting from Windows 8 and Windows Server 2012);
last written timestamp of a base block in a hive (before Windows 8.1 and Windows Server 2012 R2);
last reorganized timestamp (starting from Windows 8 and Windows Server 2012).
Also See
Windows Registry
Windows 9x Registry File (creg)
External Links
Windows NT Registry File (REGF) format, by the libregf project
Windows registry file format specification, by Maxim Suhanov, 2015-2016
How to intercept ata commands using aoe -
ATA over Ethernet (AoE) is a network protocol designed for accessing ATA storage devices over Ethernet networks.
This "How To" explains the process of using ATA over Ethernet protocol to intercept ATA commands from a Linux system (called Linux client) using Wireshark for diagnostic and testing purposes.
Linux server - Linux client configuration
First, make sure that Linux server and Linux client are connected to the same network and can exchange packets with each other.
Server configuration
Make sure that your server has vblade tool installed. Then run the following command:
# vblade 0 0 eth0 /dev/sda
Where:
eth0 - your network interface;
/dev/sda - a block device you want to be accessible through a network.
Client configuration
Make sure that your Linux client has aoe tools installed. Then run the following command:
# modprobe aoe
Make sure your vblade device is available on the client by running:
# aoe-stat
The following block devices would be available:
/dev/etherd/e0.0 - the whole disk (corresponds to /dev/sda on the server);
/dev/etherd/e0.0p1 - first partition on the disk (corresponds to /dev/sda1 on the server);
/dev/etherd/e0.0p2 - second partition on the disk (corresponds to /dev/sda2 on the server);
etc.
Intercepting ATA commands
Launch Wireshark on a client and start a capture on eth0. You can apply the following display filter to analyse AoE packets only:
eth.type == 0x88a2
Gurls -
Gruls is a bash script and is short for grep urls :
#!/bin/bash protocol="(ftp|http|https|gopher|mailto|pop|smtp|news|nntp|telnet|whois|file|imap|prospero|peercast|ed2k|irc|aim|mime|ftam|pnm|rtsp|ldap)" ip4="([1-9][0-9]?|1[0-9]{2}|2[0-4][0-9]|25[0-4])\.((0|[1-9][0-9]?|1[0-9]{2}|2[0-4][0-9]|25[0-4])\.){2}([1-9][0-9]?|1[0-9]{2}|2[0-4][0-9]|25[0-4])" fqdn="(\w(-?\w+)*\.)+[a-z]{2,}" host="(${ip4}|${fqdn})" port="(:[0-9]+)?" urlregexp="${protocol}://${host}${port}/?"
( if [ "$1" ] then while [ "$1" ] do egrep -o "$urlregexp" "$1" shift done else egrep -o "$urlregexp" /dev/stdin fi ) | sed 's;/$;;g'
Once saved in /usr/local/bin/gurls and made ​​executable, gurls can be used like this :
root@forensic# gurls a.file an.other.file https://forensics.wiki
root@forensic# strings /mnt/forensic/partition/pagefile.sys | gurls | sort | uniq -c | sort -n 10https://forensics.wiki
root@forensic# strings /dev/sdb1 | gurls > /tmp/urls
See Also
bulk_extractor provides similar functionality but on a much larger scale. Still, scripts like gurls are good for quickly searching through data.
Windows registry xml -
Microsoft's .reg format for representing MS Registry entries has many limitations, such as the inability to represent where registry information physically resides on the disk and the difficulty in representing Unicode. As a result, a variety of approaches have been implemented. Currently DFXML uses the RegXML standard to represent Registry entries.
See Also
There are several open source programs that use XML to represent the Windows Registry:
registryasxml is a Windows GUI program that exports and imports section of the Windows Registry as XML-foramtted files.
RegXML is also a Windows command-line utility that exports sections of the Windows Registry as XML-formatted files.
hivexml is a command-line utility that is part of Red Hat's libguestfs that converts Registry hives to XML.
Tracking Computer Use with the Windows® Registry Dataset, Doug White, NIST.
The complete set of code and and a WiReD XML difference set for steganographic applications.
There is one commercial program that we have found:
ComponentSource has a \$195 .NET too that allows management, importing and exporting of the registry via XML.
Serial port monitoring -
Serial Port Monitoring
Overview
Examiners should be aware how all the tools on the market work. It is crucial not to change the evidence that may reside on a cell phone, PDA, or other small scale digital device. Forensic examinations on cell phones are currently not possible if used with a USB write blocker like in computer forensics. How do examiners know the evidence is not being changed when "forensic" tools are used to extract the data from a cell phone.
A step in the right direction is to monitor the data being transferred in between the phone and the forensic workstation. Most of the devices make use of a USB cord to transfer the data. It would be advised to monitor that port to see what commands the workstation is asking the cell phone to execute. A serial port monitor would be able to independently monitor the data transferred.
Cell phone manufacturers use different command languages to communicate with the cell phone. Three of the most common types of commands are AT Commaands, Obex, and Jtag. By using a serial port monitor it may be able to see what commands the "forensic tool" is sending to the cell phone to execute. It can also be used to see if the tool automatically filters out any information that the examiner might actually need for the investigation.
Tools Here are some tools used to monitor the data transfer between COM devices.
AGGSoftware
HHD USB/Serial Port Monitoing Software
Serial Port Monitoring tool
Kismet -
Kismet is an 802.11 layer2 wireless network detector, sniffer, and intrusion detection system.
Overview
Wireshark/Tcpdump compatible data logging;
Airsnort compatible weak-iv packet logging;
Network IP range detection;
Built-in channel hopping and multicard split channel hopping;
Hidden network SSID decloaking;
Graphical mapping of networks;
Client/server architecture allows multiple clients to view a single Kismet server simultaneously;
Manufacturer and model identification of access points and clients;
Detection of known default access point configurations;
Runtime decoding of WEP packets for known networks;
Named pipe output for integration with other tools, such as a layer3 IDS like Snort;
Multiplexing of multiple simultaneous capture sources on a single Kismet instance;
Distributed remote drone sniffing;
XML output;
Over 20 supported card types.
Intrusion Detection
Kismet will detect following events:
Active network scanning (NetStumbler, PocketStumbler, etc);
SSID brute force attempts;
Broadcast disconnect/deauthenticate attacks;
Deauthenticate/disassociate flood;
Fake APs (new AP on another channel, invalid BSS timestamps);
Many DoS attacks (zero-length SSID, over-long SSID, etc).
How to ship drives -
I was recently asked about shipping drives for forensic analysis, and since I've seen this done successfully and seen failures, I thought I would write this advice up for general consumption.
Shipping disks is tricky, but often needs to be done. Copying entire images over networks is often impossible due to the sheer size of the image. If you must ship disks, here are some instructions:
Never ship the original drive (unless necessary for legal reasons). Regardless of the ultimate disposition of the original drive, always start by shipping an image.
Use one of the Write Blockers mentioned on this page. The write blocker goes on the original drive to ensure that nothing is changed on the drive by the host OS of the imaging workstation.
A drive can be imaged by a number of free software tools, such as FTK Imager. Some of these tools compress and/or encrypt data; I recommend both, as you don't want someone getting their hands on your data, and if you have multiple images to send, you can put them on one, say, 1TB drive.
Image to a bare (internal) hard drive, such as these internal hard drives.
To write to the image drive use either a Firewire/USB2/eSATA docks or USB2/eSATA docks. I recommend getting two of these.
Keep the original drive safe and intact (and unshipped) until you verify that the image has been received intact.
In addition, keep a copy of the images, if possible. This will double your image storage needs, but if you are doing this regularly, you don't want to have to re-image the drives, and I guarantee that failures will occur if you do this long enough. Also, you may have concerns about this original drive, as it may fail.
ALWAYS ALWAYS ALWAYS SHIP PROPERLY! Don't ship a drive in an enclosure, as these provide minimal protection, and the drive interface is plugged into something that can move. Not a good idea. Take the drives out of the enclosure (if there is one - if you've followed my advice above, you're working with images on bare drives). Then get either a hard drive shipping kit, if you are shipping multiple drives, or a single drive case. If you can't do this, wrap the drive in foam sheeting or bubble wrap (three times is good), and tape up the wrapped drive. If you don't have access to any other packing material, then use towels or other thick cloth. The key is something to absorb impact when the package is inevitably dropped.
Make sure you properly unmount the drive from the imaging station before you disconnect it. If you don't know how
Ejecting Media on a Mac
Ejecting Media on Windows
Ejecting Media on Linux
NB 1: This is advice, not a set of rules. Do what you can to secure your media and data.
NB 2: This does NOT take account of chain-of-evidence issues, obviously. This is about imaging and packing ONLY.
Zune -
Zune
The Zune is a brand of portable digital media player designed and marketed by Microsoft Corporation. The release date of this product in the United States is November 14, 2006 with a price of \$249.99 USD. Zune comes standard with a 30 Gigabyte(GB) hard drive builtin. In addition, a FM tuner is builtin for allowing of reception of local radio stations. It has the feature of wirelessly sharing selected full-length sample tracks, playlists, pictures, self made tracks directly from Zune to Zune.
Portable network graphics (png) -
Portable Network Graphics (aka PNG) is an image file format developed to replace GIF images on the web. It uses lossless compression and is thus not ideal for photographs. The types of images ideally suited for PNG are line-art, text, other graphics that have sharp transitions.
Format
A PNG file comprises "chunks" of data, some of which are mandatory and others of which are ancillary. Some ancillary chunks can contain metadata text or timestamps, but does not contain nearly as much metadata as exif images.
PNG files begin with the following 8-byte (hexadecimal) signature:
89 50 4E 47 0D 0A 1A 0A 
See Also
Tools for extracting image metadata
External Links
Wikipedia article
W3C Recommendation
Windows xml event log (evtx) -
The Windows XML Event Log (EVTX) format was introduces in Windows Vista as a replacement for the Windows Event Log (evt) format.
Event Viewer
On Windows the event logs can be managed with "Event Viewer" (eventvwr.msc) or "Windows Events Command Line Utility" (wevtutil.exe). Event Viewer can represent the EVTX files in both "general view" (or formatted view) and "details view" (which has both a "friendly view" and "XML view"). Note that the formatted view can hide significant event data that is stored in the event record and can be seen in the detailed view.
If you export an event log from Event Viewer additional "display information" can be exported. This display information is stored in a corresponding file named:
LocaleMetaData\%FILENAME%_%LCID%.MTA 
Where LCID is the "locale identifier" 1.
Location
C:\Windows\system32\winevt\Logs 
See Also
Windows Event Log (evt)
Windows
External Links
Mute Sysmon - Silence Sysmon via event manifest tampering, by SecurityJosh, April 23, 2020
File Format
EventLog Remoting Protocol Version 6.0 Specification, by Microsoft
Simple BinXml Example, by Microsoft
Introducing the Microsoft Vista Event Log File Format, by Andreas Schuster, in 2007
Windows XML Event Log (EVTX) format, by the libevtx project
Windows Vista/2008
Description of security events in Windows Vista and in Windows Server 2008
Windows 7
Core OS Events in Windows 7, Part 1
Core Instrumentation Events in Windows 7, Part 2
Tools
libevtx
log2timeline
wevtutil
LogParser
python-evtx
winlast
Event log explorer
Event-Log Hunting tools collection by Renzon
Context triggered piecewise hashing -
Context Triggered Piecewise Hashing, also called Fuzzy Hashing, can match inputs that have homologies. Such inputs have sequences of identical bytes in the same order, although bytes in between these sequences may be different in both content and length.
CTPH was originally based on the work of Dr. Andrew Tridgell and a spam email detector called SpamSum. This method was adapted by Jesse Kornblum and published at the DFRWS conference in 2006 in a paper Identifying Almost Identical Files Using Context Triggered Piecewise Hashing.
Implementations
ssdeep is a cross-platform CTPH client.
See Also
Piecewise hashing
External Links
SpamSum