hemu1438/forensicsgpt-training-data huggingface dataset:
Lnk -
Microsoft Windows Shortcut Files
File Format
The Windows Shortcut file has the extension .lnk. It basically is a metadata file, specific for the Microsoft Windows platform and is interpreted by the Windows Shell. The file format indicates that these files contain a specific signature, 0x4C (4C 00 00 00) at offset 0 within the file/stream. Further, the GUID (CLSID) 00021401-0000-0000-c000-000000000046 stored at byte offset 4 makes a good identifier.
Understanding this file format can be extremely useful for an analyst, as not only are shortcut files employed from (at least) Windows 95 through Windows 10, but the binary format is also used in the numbered streams within *.automaticDestinations-ms and *.customDestinations-ms Jump Lists files on Windows 7 and later.
MAC times of the target. These are a snapshot of the target date and timestamps before it was last opened. The target can be several things like for example a (linked) file;
Linked file information: Creation time : Jul 26, 2009 14:44:34 UTC Modification time : Jul 26, 2009 14:44:34 UTC Access time : Aug 12, 2010 06:41:50 UTC Local path : C:\Program Files (x86)\Windows Live\Mail\wlmail.exe 
The Shell Item list of the target;
The size of the target when it was last accessed;
Serial number of the volume where the target was stored;
Useful for correlating a USB drive or other removable media (if you can get the volume serial number off it) to a particular user or system.
Network volume share name;
Read-only, hidden, system, volume label, encryption, sparse, compressed, offline and several other target attributes;
MAC address of the host computer (sometimes);
Distributed link tracking information, e.g.
Distributed link tracker information: Machine identifier string : mysystem Droid volume identifier : 11111111-2222-3333-4444-555555555555 Droid file identifier : aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee Birth droid volume identifier : 11111111-2222-3333-4444-555555555555 Birth droid file identifier : aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee 
Notes
Windows Shell commands:
%windir%\explorer.exe ::{7007ACC7-3202-11D1-AAD2-00805FC1270E} %windir%\explorer.exe shell:::{3080F90D-D7AD-11D9-BD98-0000947B0257} 
https://docs.rainmeter.net/tips/launching-windows-special-folders/
External Links
The Meaning of Linkfiles In Forensic Examinations, by Harry Parsonage, September 2008
MS-SHLLINK
Windows Shortcut File (LNK) format, by the liblnk project
Evidentiary Value of Link Files, by Nathan Weilbacher, March 2006
Tools
Free (Non Open Source)
Windows File Analyzer, free tool that is capable of reading and reporting on Windows shortcut files
Open Source
jafat, free tool (in PERL) that is capable of reading and reporting on Windows shortcut files
liblnk
lnk-parser
lnk parser in C# that parses all structures
LECmd command line tool
Windows encrypted file system -
Encrypted File System (EFS) is the Cryptographic File System that is build into Microsoft Windows.
Windows can encrypt files on an EFS volume by file, by directory, or by the entire volume. Encryption is done using a certificate. The certificate itself is saved on the encrypted volume, but it is encrypted with a password. Volumes can be configured so that they can be recovered using one of several certificates — for example, a recovery certificate belonging to the organization that owns the computer.
How it works
The first time EFS is used Windows creates a symmetric File Encryption Key (FEK). Windows then creates an RSA public/private key pair that is used to encrypt the FEK. The private key is then encrypted with a hash of the user's passphrase and username. The FEK can also be encrypted with the organization's public key. Microsoft calls this second key a "Recovery Agent".
In Windows 2000 the computer's administrator is the default recovery agent and can decrypt all files encrypted with EFS.
In Windows XP and beyond there is no default recovery agent.
EFS can be used in conjunction with BitLocker if desired.
Recovering an EFS Key
Several tools are available that can recover an EFS key or volume if the original encryption key (or passphrase) are lost. These include:
EFS key from Passware;
Advanced EFS Data Recovery from Elcomsoft (http://www.elcomsoft.com/aefsdr.html);
EnCase Forensic (can perform a brute-force attack on the user's passphrase);
WinHex Forensic (can also perform a brute-force attack on the user's passphrase).
Linux
It is possible to decrypt files using ntfsdecrypt tool. In this case, you should get the private key first (by running cipher /x filename.pfx on a Windows system).
Disk image -
A disk image is a full disk copy of the data making up the partition table, file allocation tables and data partitions without regard for operating system.
A disk image should be made prior to performing any forensic analysis of the disk. Creating a disk image is important in forensics for several reasons:
1. Ensure that disk information is not inadvertantly changed during analysis.
2. By performing an original disk image and storing the original disk, it is possible to reproduce forensic test results with an exact reproduction of analysis methods on the original evidence.
3. Disk imaging will capture information invisible to the operating system in use (e.g. hidden partitions, ext3 partitions on a Windows machine, etc.)
Software
Popular software used to create disk images includes Norton Ghost copy of the original media should be done using the software, which may not be the default settings on the software.
Other possible software, programs include:
dd
dcfldd
EnCase
FTK
Metadata -
Metadata is data about data. Metadata plays a number of important roles in computer forensics:
It can provide corroborating information about the document data itself.
It can reveal information that someone tried to hide, delete, or obscure.
It can be used to automatically correlate documents from different sources.
Since metadata is fundamentally data, it suffers all of the data quality and pedigre issues as any other form of data. Nevertheless, because metadata isn't generally visible unless you use a special tool, more skill is required to alter or otherwise manipulate it.
Some kinds of metadata that are interesting in computer forensics:
File system metadata (e.g. MAC times, access control lists
Digital image metadata. Although information such as the image size and number of colors are technically metadata, JPEG and other file formats store additional data about the photo or the device that acquired it.
Document metadata, such as the creator of a document, it's last print time, etc.
Below are some common data and metadata formats, the files in which they are found, and a collection of tools that can be used to extract information.
Exchangeable Image File (EXIF) (JPEG and TIFF image files; Music Files) The EXIF format describes a format for a block of data that can be embedded into JPEG and TIFF image files, as well as RIFF WAVE audio files. Information includes date and time information, camera settings, location information, textual descriptions, and copyright information.
LibExif
Adroit Photo Forensics
ID3 (MP3 files) Implemented as a small block of data stored at the end of MP3 files. ID3v1 is a 128-byte block in a specified format allowing 30 bytes for song, artist and album, 4 bytes for year, 30 bytes for comment, and 1 byte for genre. ID3v1.1 adds a track number. ID3v2 is a general container structure.
id3lib, a widely-used open source C/C++ ID3 implementation.
MP3::Info (Perl)
MPEG::ID3v2Tag (Perl)
Microsoft OLE Compound File Microsoft Office document files contain a huge amount of metadata. They are created as OLE Compound Files and mainly stored in the so called property set streams. Here are some tools for processing them:
Jakarta POI Open Source implementation in Java.
SquirrelGripper Perl script Extract fields into SQLite Database.
FOCA - Eleven Paths Scan and group metadata information.
TIFF The Tagged Image File Format allows one or more images to be bundled in a single file. Multiple compression formats are supported. EXIF files can be stored inside TIFFs.
LibTIFF
TIFF FAQ
See Also
Document_Metadata_Extraction
External links
Wikipedia: Metadata
Automated Metadata Extraction, by James Migletz, June 2008
Why is Microsoft Office metadata what it is, by Corey Harrell
Xmount -
Description taken from xmount project site on Pinguin's HQ:
xmount allows you to convert on-the-fly between multiple input and output harddisk image types. xmount creates a virtual file system using FUSE (Filesystem in Userspace) that contains a virtual representation of the input image.
The virtual representation can be in:
raw DD
DMG
VHD
VirtualBox's virtual disk file format
VmWare's VMDK file format
Input images can be:
raw DD
EWF (Expert Witness Compression Format)
Advanced Forensics Format
In addition, xmount also supports virtual write access to the output files that is redirected to a cache file. This makes it possible to boot acquired harddisk images using QEMU, KVM, VirtualBox, VmWare or alike.
See Also
Disk Images
External Links
Pinguin's HQ
HOWTO: Booting an acquired hard disk image containing a Windows installation using xmount and OpenGates.
Xmount: When "Changing" the Evidence isn't so Bad, by John Lehr, March 21, 2013
Ftk imager -
The Forensic Toolkit Imager (FTK Imager) is a commercial forensic imaging software package distributed by AccessData.
FTK Imager supports storage of disk images in EnCase's or SMART's file format, as well as in raw (dd format. With Isobuster technology built in, FTK Imager Images CD's to a ISO/CUE file combination. This also includes multi and open session CDs.
Notes
Important: when using FTK Imager to create a forensic image of a suspect's hard drive, make sure you are using a hardware-based write blocking device. This ensures that your operating system does not alter the suspect's hard drive when you attach the drive to your computer.
Joachim metz -
Joachim Metz is a Digital researcher, IT/IS specialist.
In 2006 he started working in the field of computer forensics as a digital forensic investigator at Hoffmann Investigations. At that time Hoffmann Investigations carried out digital forensic investigations for organisations (private law).
Before that he worked in multiple Information Communication Technology (ICT) disciplines like: system and network administration, programming, deployment, etc. and also Information Security (IS). He has been working in the field of digital forensics for several years now.
Philosophy warning
In his opinion digital forensic investigators should be transparent in both their findings and methods. The statement "the tool provided me with the evidence" just does not cut it. In his work he experienced a lot of serious errors in 'digital forensic software' and corresponding human interpretation, e.g. tools that represent the FAT access date as a date and time value and is interpreted as such. He has put a lot of effort in providing alternatives and means to verify findings by breaking open file formats and improving file recovery methods.
Ancient history alert
For him breaking open file formats dates back to Might and Magic 3 save games and recovering deleted and corrupted files under DOS using PCTOOLS.
Marketing alert
Some of his work are file format libraries like libyal.
Also a proof-of-concept carving tool called ReviveIt (revit), which keeps surprising him because of its versatile application even in recovering NTFS-compressed files.
The challenge he likes about the field of digital forensics is that there is a lot out there still to be discovered ;-)
Aimage -
aimage (the advanced imager) was an imaging tool that was part of AFF.
aimage can create files in raw, AFF, AFD, or AFM formats. AFF and AFD formats can be compressed or uncompressed. aimage can optionally compress and calculate MD5 or SHA-1 hash residues while the data is being copied. It had intelligent error recovery, similar to what is in ddrescue.
'aimage was withdrawn from support (December 25, 2010)
Linux distributions that packaged image are encouraged to drop it and package guymager instead.
See Also
How_to_image_an_IDE_disk_with_aimage_and_FreeBSD
Afconvert -
afconvert is part of the AFF toolset. This program can interconvert between all of the different file formats that AFF supports. It can also be used to restore AFF files on raw disk partitions. The remainder of this article uses the term "ISO/RAW" to describe a bit-for-bit disk image with no metadata. This format is also called a "dd image".
Converting ISO/RAW to AFF
To convert a file from ISO to AFF, use this command.
% afconvert filename.iso
This will give you output like this:
convert filename.iso --> filename.aff Converting page 1 of 6 Converting page 2 of 6 Converting page 3 of 6 Converting page 4 of 6 Converting page 5 of 6 Converting page 6 of 6
md5: 949442a1cd26d3d26b28a529e08e32df sha1: 198dd3779dce1de0859183a619dd1a9ffbc249f7 bytes converted: 108580864 Total pages: 7 (3 compressed)
Converting AFF to ISO/RAW
% afconvert -r filename.aff Convert filename.aff --> filename.raw Converting page 1 of 6 Converting page 2 of 6 Converting page 3 of 6 Converting page 4 of 6 Converting page 5 of 6 Converting page 6 of 6 md5: 949442a1cd26d3d26b28a529e08e32df sha1: 198dd3779dce1de0859183a619dd1a9ffbc249f7 bytes converted: 108580864 Conversion finished.
Other formats
afconvert can convert to other formats as well. For example, you can convert an AFF file (which stores all of the data and metadata in a single file) to AFM (which stores the metadata in an AFM file and the data in a splitraw file):
% afconvert -a afm filename.aff
convert filename.aff --> filename.afm Converting page 6 of 6 md5: 949442a1cd26d3d26b28a529e08e32df sha1: 198dd3779dce1de0859183a619dd1a9ffbc249f7 bytes converted: 108580864 Conversion finished.
02:40 PM s:~$ ls -l filename* -rw-r--r-- 1 simsong simsong 108580864 Mar 30 14:40 filename.000 -rwxr-xr-x 1 simsong simsong 108528960 Mar 30 13:42 filename.aff* -rwxr-xr-x 1 simsong simsong 1463 Mar 30 14:40 filename.afm* 02:40 PM s:~$
In this example, filename.aff is the original AFF file, filename.afm is the metadata and filename.000 is the first splitraw file. Here there is only one splitraw file, and the file didn't expand much because the file is a disk image of a file that contains a ZIP file.
Restoring AFF onto a RAW partition
You can use afconvert to restore an AFF file onto a raw partition. In this example, /dev/md1 is a raw partition of a memory file system on a computer running FreeBSD:
# mdconfig -a -t malloc -s 10858k md1 #
We can restore filename.aff onto /dev/md1:
Affuse -
affuse is a Filesystem in Userspace (FUSE) implementation that allows AFF files to appear as raw files on Linux systems. This allows the files to be processed with forensic tools that do not understand AFF. It also allows the disk images to be actually mounted, should you want to do that.
For example, say that /project/0089.aff is a disk image that you wish to process with raw tools. With affuse you can mount it like this:
# affuse /project/00709.aff /mnt/p # ls /mnt/p 0070.aff.raw
This raw file could then be mounted:
# mount -t vfat -o loop=/dev/loop0,ro,noexec,offset=32256 /mnt/p/0070.aff.raw /mnt/r # ls /mnt/r total 316 -rwxr-xr-x 1 root root 93880 May 11 1998 command.com -r-xr-xr-x 1 root root 222390 May 11 1998 io.sys -r-xr-xr-x 1 root root 9 May 11 1998 msdos.sys
External Links
FUSE
Ubuntu manpage
Aff development task list -
AFFLIB has been depreciated. As a result, this page is now obsolete.
High Priority
When afinfo -a is run on a non-AFF file, it notes it is a "Raw" file, but continues to attempt to process segments. It should exit if it does not find valid AFF segments. For example, running it against a raw image of a 40GB disk created with aimage, afinfo -a reported 2,386 segments then finished with the error message "Cannot calculate missing pages."
The library does not compile on 64-bit versions of Fedora Core 7 Linux.
Create man pages and/or documentation for AFF toolkit. To wit:
* aimage * ident * afcat * afcompare * afconvert * affix * affuse * afinfo * afstats * afxml * afsegment
Add a usage description to afcat. When run with no arguments the output should say what the program does.
Create man pages and/or documentation for AFF library functions (e.g. ,af_open, af_get_imagesize)
Build library as a shared library using libtool. This will allow developers using the library to just link to the AFF. Without it, developers must link to the static library and the individual libraries necessary on that machine. There is no good way to determine those extra libraries.
Document that af_write may not be called without first setting the image_pagesize value inside of the AFFILE structure. Not doing so causes a divide by zero error. Perhaps we should 1. Check that image_pagesize is not zero and 2. Set image_pagesize to a known good default value when opening a new AFF file for writing.
Check aimage ability to write a file of 1,073,741,825 bytes ((2**30)+1). Correctly reported reading/writing a file that was a 1,073,741,824 random byte stream, but did not pick up the extra byte when it was added to the file. ls -la correctly shows the size with the extra byte. Also, added 42 additional bytes which were not apparently read or written. UPDATE - With 511 bytes added, still didn't read/write full file, however, adding 512 bytes did cause the whole file (1,073,742,336 bytes) to be read/written.
Medium Priority
Is there a set of segment names that must be defined to have a valid AFF file?
Document that af_open (when writing a file) does more than a standard fopen command. The command writes an AFF stub of some kind to the output file. Users should be cautioned not to use this function as a test, lest they overwrite data.
Does af_open refuse to open a file for writing if it already exists? If so, what kind of error does it return?
Document how to programmatically enumerate all segments and values in a file. That is, explain how to get the output of $ afinfo -a.
Low Priority
Add library function to open standard input. Perhaps:
AFFILE * af_open_stdin(void);
Cue sheet format -
The CUE sheet format (.cue) file is often used in combination with .bin or .iso file(s) to define the track and/or session layout of an optical disc.
Contents
The CUE sheet is a text based format that contains commands which specify the layout.
History
The CUE sheet originates from CDRWIN, but various additions have been made like the CDTEXT and IsoBuster extensions.
See also
Wikipedia on CUE sheet
CUE sheet format
Disk images -
There are various types of disk image formats. For clarity the formats are divided by means of their original purpose, e.g. were they intended to be used in (disk) forensics or virtualization.
Disk images are usually block by block copies of the original data source. Depending on the data source, the blocks can be bytes, sectors, clusters, pages or similar. The disk image format may be further enhanced with additional information, such as error detection, error correction, data digest (hash, and compression.
Forensics Disk Image formats
Forensics File Formats
Advanced Forensics Format (aff)
Advanced Forensic Framework 4 (aff4)
Expert Witness Compression Format (ewf)
gfzip
ProDiscover image file format
Raw Image Format
sgzip
Virtualization Disk Image formats
Open Virtualization Format (ovf)
Parallels Virtual Hard Disk Format
QCOW Image Format
Virtual Disk Image (vdi)
Virtual Hard Disk (vhd)
VMWare Virtual Disk Format (vmdk)
Other Disk Image formats
Mac OS X has support for various disk image types build-in, some of which are:
read-write disk image (.dmg): raw, UDIF, NDIF
Sparse disk image (.spareimage)
Sparse bundle disk image (.sparsebundle)
Windows
Windows Imaging File Format (wim)
Dc3dd -
dc3dd is a patched version of GNU dd with added features for computer forensics. It was developed at the DoD Cyber Crime Center by Jesse Kornblum. The first release, corresponding to Coreutils version 6.9.91, was published on 1 Feb 2008.
Comparison to GNU dd
The following features are available in dc3dd that are not found in GNU dd:
On the fly hashing with multiple algorithms (md5, SHA-1, SHA-256, and SHA-512) with variable sized piecewise hashing
Able to write errors directly to a file
Combined error log. Groups errors together (e.g. Had 1,023 'Input/ouput errors' between blocks 17-233')
Pattern wiping. Wipe output files with a single hex digit or a text pattern
Verify mode
Progress reports. See the progress of the operation while it's running
Split output. Able to split output files into fixed size chunks
The following changes to GNU dd's behavior were made:
On a partial read, the whole block is wiped with zeros. This allows for repeatable reads/hashes of a drive with errors.
Comparison to dcfldd
Although there are definitely similarities between dc3dd and dcfldd, the programs are based on slightly different code bases and have different feature sets. dcfldd is a fork of GNU dd, whereas dc3dd is a patch to the current version. This means that dc3dd will be updated every time GNU dd is updated, whereas dcfldd has its own release schedule. Certain features added to GNU dd after dcfldd forked, such as direct input/output mode, are not found in dcfldd.
On the other hand, dcfldd supports more hashing algorithms than dc3dd, allows the user greater control over how hashes are displayed, supports wiping output files with random patterns, and is supported on the Cygwin platform.
See Also
dcfldd
External Links
Official website
Sourceforge project page
Dd rescue -
dd_rescue, is an an advanced evolution of dd, a command line program that has been ported only for UNIX/Linux. The program uses a complex series of flags to allow the user to image or write data from and to raw image files. Like dcfldd, the program makes an effort to keep the user apprised of the status of the current operation.
ddrescue and dd_rescue are completely different programs which share no development between them. The two projects are not related in any way except that they both attempt to enhance the standard dd tool and coincidentally chose similar names for their new programs.
Sample usage
Here is a common dd_rescue command:
UNIX/Linux
$ dd_rescue /dev/hda myfile.img 
STDOUT
A large difference between ddrescue and dd_rescue is that dd_rescue can pipe output to STDOUT whereas ddrescue can only transfer block/file to block/file.
One example of this usage would be transfering a disk image over the network using ssh.
dd_rescue /dev/sda1 - | ssh user@remote.host "cat - > /remote/destination/file.img" 
Another example would be adding compression to an image file on the fly.
dd_rescue /dev/sda1 - | bzip2 > /dir/file.img.bz2 
Cautions
Unlike regular dd, dd_rescue does not use the command line arguments if or of.
See also
aimage
Blackbag
dcfldd
dd
ddrescue