deardaniel/secdata-raw huggingface dataset:
Log2timeline timeline generation: plaso.storage

plaso.storage package
=====================

Subpackages
-----------

.. toctree::
   :maxdepth: 4

   plaso.storage.fake
   plaso.storage.redis
   plaso.storage.sqlite

Submodules
----------

plaso.storage.factory module
----------------------------

.. automodule:: plaso.storage.factory
   :members:
   :show-inheritance:
   :undoc-members:

plaso.storage.logger module
---------------------------

.. automodule:: plaso.storage.logger
   :members:
   :show-inheritance:
   :undoc-members:

plaso.storage.reader module
---------------------------

.. automodule:: plaso.storage.reader
   :members:
   :show-inheritance:
   :undoc-members:

plaso.storage.serializers module
--------------------------------

.. automodule:: plaso.storage.serializers
   :members:
   :show-inheritance:
   :undoc-members:

plaso.storage.time\_range module
--------------------------------

.. automodule:: plaso.storage.time_range
   :members:
   :show-inheritance:
   :undoc-members:

plaso.storage.writer module
---------------------------

.. automodule:: plaso.storage.writer
   :members:
   :show-inheritance:
   :undoc-members:

Module contents
---------------

.. automodule:: plaso.storage
   :members:
   :show-inheritance:
   :undoc-members:
Log2timeline timeline generation: plaso.analysis

plaso.analysis package
======================

Submodules
----------

plaso.analysis.bloom module
---------------------------

.. automodule:: plaso.analysis.bloom
   :members:
   :show-inheritance:
   :undoc-members:

plaso.analysis.browser\_search module
-------------------------------------

.. automodule:: plaso.analysis.browser_search
   :members:
   :show-inheritance:
   :undoc-members:

plaso.analysis.chrome\_extension module
---------------------------------------

.. automodule:: plaso.analysis.chrome_extension
   :members:
   :show-inheritance:
   :undoc-members:

plaso.analysis.definitions module
---------------------------------

.. automodule:: plaso.analysis.definitions
   :members:
   :show-inheritance:
   :undoc-members:

plaso.analysis.hash\_tagging module
-----------------------------------

.. automodule:: plaso.analysis.hash_tagging
   :members:
   :show-inheritance:
   :undoc-members:

plaso.analysis.interface module
-------------------------------

.. automodule:: plaso.analysis.interface
   :members:
   :show-inheritance:
   :undoc-members:

plaso.analysis.logger module
----------------------------

.. automodule:: plaso.analysis.logger
   :members:
   :show-inheritance:
   :undoc-members:

plaso.analysis.manager module
-----------------------------

.. automodule:: plaso.analysis.manager
   :members:
   :show-inheritance:
   :undoc-members:

plaso.analysis.mediator module
------------------------------

.. automodule:: plaso.analysis.mediator
   :members:
   :show-inheritance:
   :undoc-members:

plaso.analysis.nsrlsvr module
-----------------------------

.. automodule:: plaso.analysis.nsrlsvr
   :members:
   :show-inheritance:
   :undoc-members:

plaso.analysis.sessionize module
--------------------------------

.. automodule:: plaso.analysis.sessionize
   :members:
   :show-inheritance:
   :undoc-members:

plaso.analysis.tagging module
-----------------------------

.. automodule:: plaso.analysis.tagging
   :members:
   :show-inheritance:
   :undoc-members:

plaso.analysis.test\_memory module
----------------------------------

.. automodule:: plaso.analysis.test_memory
   :members:
   :show-inheritance:
   :undoc-members:

plaso.analysis.unique\_domains\_visited module
----------------------------------------------

.. automodule:: plaso.analysis.unique_domains_visited
   :members:
   :show-inheritance:
   :undoc-members:

plaso.analysis.viper module
---------------------------

.. automodule:: plaso.analysis.viper
   :members:
   :show-inheritance:
   :undoc-members:

plaso.analysis.virustotal module
--------------------------------

.. automodule:: plaso.analysis.virustotal
   :members:
   :show-inheritance:
   :undoc-members:

Module contents
---------------

.. automodule:: plaso.analysis
   :members:
   :show-inheritance:
   :undoc-members:
Log2timeline timeline generation: plaso.containers

plaso.containers package
========================

Submodules
----------

plaso.containers.analysis\_results module
-----------------------------------------

.. automodule:: plaso.containers.analysis_results
   :members:
   :show-inheritance:
   :undoc-members:

plaso.containers.analyzer\_result module
----------------------------------------

.. automodule:: plaso.containers.analyzer_result
   :members:
   :show-inheritance:
   :undoc-members:

plaso.containers.artifacts module
---------------------------------

.. automodule:: plaso.containers.artifacts
   :members:
   :show-inheritance:
   :undoc-members:

plaso.containers.counts module
------------------------------

.. automodule:: plaso.containers.counts
   :members:
   :show-inheritance:
   :undoc-members:

plaso.containers.event\_sources module
--------------------------------------

.. automodule:: plaso.containers.event_sources
   :members:
   :show-inheritance:
   :undoc-members:

plaso.containers.events module
------------------------------

.. automodule:: plaso.containers.events
   :members:
   :show-inheritance:
   :undoc-members:

plaso.containers.plist\_event module
------------------------------------

.. automodule:: plaso.containers.plist_event
   :members:
   :show-inheritance:
   :undoc-members:

plaso.containers.reports module
-------------------------------

.. automodule:: plaso.containers.reports
   :members:
   :show-inheritance:
   :undoc-members:

plaso.containers.sessions module
--------------------------------

.. automodule:: plaso.containers.sessions
   :members:
   :show-inheritance:
   :undoc-members:

plaso.containers.tasks module
-----------------------------

.. automodule:: plaso.containers.tasks
   :members:
   :show-inheritance:
   :undoc-members:

plaso.containers.warnings module
--------------------------------

.. automodule:: plaso.containers.warnings
   :members:
   :show-inheritance:
   :undoc-members:

plaso.containers.windows\_events module
---------------------------------------

.. automodule:: plaso.containers.windows_events
   :members:
   :show-inheritance:
   :undoc-members:

Module contents
---------------

.. automodule:: plaso.containers
   :members:
   :show-inheritance:
   :undoc-members:
Log2timeline timeline generation: plaso.storage.redis

plaso.storage.redis package
===========================

Submodules
----------

plaso.storage.redis.reader module
---------------------------------

.. automodule:: plaso.storage.redis.reader
   :members:
   :show-inheritance:
   :undoc-members:

plaso.storage.redis.redis\_store module
---------------------------------------

.. automodule:: plaso.storage.redis.redis_store
   :members:
   :show-inheritance:
   :undoc-members:

plaso.storage.redis.writer module
---------------------------------

.. automodule:: plaso.storage.redis.writer
   :members:
   :show-inheritance:
   :undoc-members:

Module contents
---------------

.. automodule:: plaso.storage.redis
   :members:
   :show-inheritance:
   :undoc-members:
Log2timeline timeline generation: plaso.parsers.cookie plugins

plaso.parsers.cookie\_plugins package
=====================================

Submodules
----------

plaso.parsers.cookie\_plugins.ganalytics module
-----------------------------------------------

.. automodule:: plaso.parsers.cookie_plugins.ganalytics
   :members:
   :show-inheritance:
   :undoc-members:

plaso.parsers.cookie\_plugins.interface module
----------------------------------------------

.. automodule:: plaso.parsers.cookie_plugins.interface
   :members:
   :show-inheritance:
   :undoc-members:

plaso.parsers.cookie\_plugins.manager module
--------------------------------------------

.. automodule:: plaso.parsers.cookie_plugins.manager
   :members:
   :show-inheritance:
   :undoc-members:

Module contents
---------------

.. automodule:: plaso.parsers.cookie_plugins
   :members:
   :show-inheritance:
   :undoc-members:
Log2timeline timeline generation: plaso.preprocessors

plaso.preprocessors package
===========================

Submodules
----------

plaso.preprocessors.generic module
----------------------------------

.. automodule:: plaso.preprocessors.generic
   :members:
   :show-inheritance:
   :undoc-members:

plaso.preprocessors.interface module
------------------------------------

.. automodule:: plaso.preprocessors.interface
   :members:
   :show-inheritance:
   :undoc-members:

plaso.preprocessors.linux module
--------------------------------

.. automodule:: plaso.preprocessors.linux
   :members:
   :show-inheritance:
   :undoc-members:

plaso.preprocessors.logger module
---------------------------------

.. automodule:: plaso.preprocessors.logger
   :members:
   :show-inheritance:
   :undoc-members:

plaso.preprocessors.macos module
--------------------------------

.. automodule:: plaso.preprocessors.macos
   :members:
   :show-inheritance:
   :undoc-members:

plaso.preprocessors.manager module
----------------------------------

.. automodule:: plaso.preprocessors.manager
   :members:
   :show-inheritance:
   :undoc-members:

plaso.preprocessors.mediator module
-----------------------------------

.. automodule:: plaso.preprocessors.mediator
   :members:
   :show-inheritance:
   :undoc-members:

plaso.preprocessors.windows module
----------------------------------

.. automodule:: plaso.preprocessors.windows
   :members:
   :show-inheritance:
   :undoc-members:

Module contents
---------------

.. automodule:: plaso.preprocessors
   :members:
   :show-inheritance:
   :undoc-members:
Log2timeline timeline generation: index

##################
User documentation
##################

.. toctree::
   :maxdepth: 3

   How to get started <Users-Guide>
   Creating a timeline <Creating-a-timeline>
   Using collection Filters <Collection-Filters>
   Event filters <Event-filters>
   Analysis plugins <Analysis-plugins>
   Tips and Tricks <Tips-and-Tricks>
   Troubleshooting <Troubleshooting>
   Feature requests and bug reports <Feature-requests-and-bug-reports>
   Log2Timeline Perl (Legacy) <Log2Timeline-Perl-(Legacy)>
Log2timeline timeline generation: Ubuntu Packaged Release

# Installing Plaso on Ubuntu

### Ubuntu 24.04 LTS (noble)

**Note that other versions of Ubuntu are not supported at this time.**

Make sure the universe apt repository is enabled.

```
sudo add-apt-repository universe
```

To install Plaso from the GIFT Personal Package Archive (PPA) add the [GIFT PPA](https://launchpad.net/~gift):

```
sudo add-apt-repository ppa:gift/stable
```

Update and install Plaso:

```
sudo apt-get update
sudo apt-get install plaso-tools
```

### SANS Investigative Forensic Toolkit (SIFT) Workstation

SIFT workstation is an independent project that provides Plaso releases. We
strongly encourage to ensure you are running the latest version of Plaso when
using SIFT.

If you are using SIFT and you have a deployment problem please report that
directly to the [SIFT project](https://github.com/teamdfir/sift).
Log2timeline timeline generation: Analysis plugin virustotal

# Virustotal analysis plugin

Notes on how to use the virustotal analysis plugin.

## Getting an API key

The virustotal analysis plugin uses the public Virustotal API, and requires an
API key to operate. The process for obtaining an API key is [here](https://developers.virustotal.com/reference#getting-started).

## Running the analysis plugin

First run log2timeline to hash files and extract events:

```bash
log2timeline.py --storage-file timeline.plaso image.raw
```

Note that hashing must be turned on for the virustotal plugin to work
correctly. This is default setting for log2timeline.py.

Next run psort to tag events:

```bash
psort.py --analysis virustotal --virustotal-api-key $API_KEY -o null timeline.plaso
```

If a file processed by Plaso is present in virustotal and has been detected as
malicious by one more detection engines, it will be tagged with
`virustotal_detections_$NUMBER_OF_DETECTIONS`. If the file is in Virustotal,
but it hasn't been fully analyzed yet, it will be tagged with
`virustotal_analysis_pending`. If the file is in Virustotal, but has not been
detected as malicious, it will be tagged with `virustotal_no_detections`. If
the file isn't in Virustotal, it will be tagged as `virustotal_not_present`.

The last step would be to export a timeline with the tags:

```bash
psort.py -o dynamic --fields datetime,timestamp_desc,source,source_long,message,parser,tag -w timeline.csv timeline.plaso
```
Log2timeline timeline generation: Using image export

# Using image_export.py

**PAGE WIP**

**image_export** is a command line tool to export file content from a storage media image or device based on various filter criteria, such as extension names, filter paths, file format signature identifiers, file creation date and time ranges, etc.

## Usage

To get a full list of parameters that can be passed to the tool use the ``-h`` or ``--help`` switch. The only required argument is ``IMAGE``, which should point to your source data.

```
image_export.py IMAGE
```

Command above will extract all unique allocated files and generate hashes.json file to ``export/`` directory, which is relative to the folder where you run ``image_export.py`` from. File ``hashes.json`` will contain list of unique hashes of extracted files. In case there are any duplicates, the file will contain paths to all duplicate files.

There are several ways to define how you want to limit the amount files to extract:
 + Based on path, filename or extension name
 + Based on time range
 + Based on format signature

### Format signatures
In order to extract files based on their signature, use flag ``--signatures``. To list all available file signatures run :

```
image_export.py --signatures list
```

To extract all files with Windows PE Binary signature use the command below:

```
image_export.py  --signatures exe_mz [IMAGE]
```

### Filename
You can filter the extracted files based on their filename. If you want extract all files with filename ``.bash_history`` use the command below:

```
image_export.py --names .bash_history [IMAGE]
```

Flag ``--names`` accept comma separated strings.

### Extension
You can extract files based on their extension. To extract all docx files run the command below:


```
image_export.py --extensions docx [IMAGE]
```

Flag ``--extensions`` accept comma separated strings.

### Time range
You can extract files, which timestamp falls into date time range provided in the command line parameters:

```
image_export.py --date-filter "crtime,2019-09-01,2019-09-30" [IMAGE]
```

Command above will extract all files created in September, 2019.


For more details on date time filtering run ``image_export.py -h``.

### Output folder
In order to specify custom output folder, where all the files will be extracted, provide ``-w`` flag:

```
image_export.py -w ~/image_export_output [IMAGE]
```

### Duplicate handling
By default image_export.py will not extract duplicate files, however paths to all duplicate files will be stored in hashes.json file. If you'd like to extract duplicate files add `` --include_duplicates`` flag.


### Collection filters
More details: [collection filters](Collection-Filters.md)


## Other options

Talk about:
```
--data
--vss_stores VSS_STORES
```
Log2timeline timeline generation: Using psteal

# Using psteal.py

**psteal** (Plaso SýndarheimsTímalína sem Er ALgjörlega sjálfvirk) is a command
line tool which uses [log2timeline](Using-log2timeline.md) and [psort](Using-psort.md)
engines to extract and process [events](Scribbles-about-events.md#what-is-an-event)
in one go.

It is a quick shortcut to the default approach and only supports a limited
subset of options the above tools provide.

## Usage

To see a list of all available parameters you can pass to psort use ``-h`` or
``--help``.

Psteal requires at least a source evidence, specified with ``--source`` and a
output with ``-w``. For example:

`psteal.py --source ~/cases/greendale/registrar.dd -w /tmp/registrar.csv` will
produce a csv file containing all the events from an image, using log2timeline
and psort defaults options.

The intermediary Plaso storage file will be created in the local directory. In
the previous example it will be named ``<TIMESTAMP>-registrar.dd.plaso``.
This can be used for further processing with Psort or [Timesketch](https://github.com/google/timesketch).

## Options

Psteal purposefully supports only a limited subset of options from both
[log2timeline](Using-log2timeline.md) and [psort](Using-psort.md) tools.

Please refer to their respective documentations for more information, for
example for help regarding the [output formats](Using-psort.md#Output).

If your use case requires specific options to either log2timeline or psort,
please use both command line tools separately.
Log2timeline timeline generation: Analysis plugin chrome extension

# Chrome extension analysis plugin

Notes on how to use the chrome_extension analysis plugin.

## Running the analysis plugin

First run log2timeline to extract events:

```bash
log2timeline.py --storage-file timeline.plaso image.raw
```

Note that the Chrome extension analysis plugin analyzes
file system data such as events with data type:

* fs:stat

Next run psort to determine Chrome extensions:

```bash
psort.py --analysis chrome_extension -o null timeline.plaso
```

This will extract information of Chrome extensions, such as name and identifier
of the extension and corresponding username. The analysis results can be
reviewed with pinfo:

```bash
pinfo.py --report chrome_extension timeline.plaso
```
Log2timeline timeline generation: Installation Problems

# Installation Problems

General advice:

* search the contents of this page and troubleshooting pages referenced
* search the issue tracker including closed issues, maybe someone else encountered the same issue before you
* search the discuss mailing list, maybe someone else encountered the same issue before you
* file and issue or ask on the discuss mailing list, **make sure to add as much relevant details as possible**

## log2timeline failing dependencies check

When I run log2timeline it warns that certain dependencies are missing e.g.
```
Checking availability and versions of plaso dependencies.
[FAILURE]       missing: pyewf.
```

**TODO: add more text**
Log2timeline timeline generation: Analysis plugin bloom

# Bloom Analysis Plugin

Notes on how to use the bloom analysis plugin.

## Prerequisite

A prerequisite to use this plugin is to have a bloom-filter database with information about hashes.

The bloom plugin uses a specific bloom-filter implementation known to be compatible with the following implementations:

* Golang: https://github.com/DCSO/bloom
* Python: https://github.com/DCSO/flor

These implementation could be used to generate a custom bloom-filter files. It is important to note that the hashes **must** be inserted in upper case.

Furthermore, bloom-filters are a probabilistic storage format. In the case of bloom, this means that:

* No false negative could occur;
* False positives can occur with a predetermined probability, defined when creating the bloom-filter file.

A bloom-filter database is available on the CIRCL.lu website: https://circl.lu/services/hashlookup/#querying-hashlookup-without-online-queries

## Running the analysis plugin

First run `log2timeline` to calculate the hashes:

```bash
log2timeline.py --hashers sha1 --storage-file timeline.plaso image.raw
```

**Make sure to enable hashers supported by the Bloom Filter, which is sha1 in this example.**

Next run `psort` to tag events:

```bash
psort.py --analysis bloom --bloom_file hashlookup-full.bloom -o null timeline.plaso
```

The last step would be to export a timeline with the tags (By default, the tag value is `bloom_present`) :

```bash
psort.py -o dynamic --dynamic-time --fields datetime,timestamp_desc,source,source_long,message,parser,data_type,display_name,tag,sha1_hash -w timeline.csv timeline.plaso
```

```bash
psort.py --analysis hashlookup_bloom --hashlookup_bloom_file hashlookup-full.bloom -o json_line -w timeline.jsonl 20221228T162736-Laptop1Final.E01.plaso
```
Log2timeline timeline generation: Creating a timeline

# Creating a timeline

## Using psteal

The quickest way to generate a timeline with Plaso is using the "psteal"
frontend. For example:

```
psteal.py --source image.raw -o dynamic -w registrar.csv
```

This will produce a CSV file containing all the events from an image, with some
sensible defaults.

## Using log2timeline and psort

Alternatively you can use "log2timeline" and "psort". For example:

```bash
log2timeline.py --storage-file timeline.plaso image.raw
psort.py -o dynamic -w registrar.csv timeline.plaso
```
Log2timeline timeline generation: Tagging Rules

# Tagging Rules

Plaso provides various configuration files for the [tagging analysis plugin](Analysis-plugin-tagging.md).

## Linux tagging rules

The Linux tagging rules are stored in the file: [tag_linux.txt](https://github.com/log2timeline/plaso/blob/main/plaso/data/tag_linux.txt)

The sections below provide more context regarding specific tagging rules.

### application_execution

This rule tags application execution events on Linux, which are defined as:

* a command from bash history
* a Docker file system layer event
* a SELinux log line where the audit type is "EXECVE"
* a command from zsh history
* a syslog line that indicates a cron task was run, for example:
```
Mar 11 00:00:00 ubuntu2015 CRON[3]: (root) CMD (touch /tmp/afile.txt)
```

* a syslog line that contains "COMMAND="

### login

### login_failed

### logout

### session_start

### session_stop

### boot

### shutdown

### runlevel

### device_connection

### device_disconnection

### application_install

### service_start

### service_stop

### promiscuous

### crash

## MacOS tagging rules

The MacOS tagging rules are stored in the file: [tag_macos.txt](https://github.com/log2timeline/plaso/blob/main/plaso/data/tag_macos.txt)

The sections below provide more context regarding specific tagging rules.

### application_execution

### application_install

### autorun

### file_download

### device_connection

### document_print

## Windows tagging rules

The Windows tagging rules are stored in the file: [tag_windows.txt](https://github.com/log2timeline/plaso/blob/main/plaso/data/tag_windows.txt)

The sections below provide more context regarding specific tagging rules.

### application_execution

### application_install

### application_update

### application_removal

### document_open

### login_failed

### login_attempt

### logoff

### session_disconnection

### session_reconnection

### shell_start

### task_schedule

### job_success

### action_success

### name_resolution_timeout

### time_change

### shutdown

### system_start

### system_sleep

### autorun

### file_download

### document_print

### firewall_change
Log2timeline timeline generation: Analysis plugin nsrlsvr

# Nsrlsvr Analysis Plugin

Notes on how to use the nsrlsvr analysis plugin.

## Setting up nsrlsvr

The source of nsrlsvr can be found [here](https://github.com/rjhansen/nsrlsvr)

Follow the [installation instructions](https://github.com/rjhansen/nsrlsvr/blob/master/INSTALL).

## Running nsrlsvr

To run nsrlsvr:

```bash
nsrlsvr -f /fullpath/NSRLFile.txt
```

To test if nsrlsvr is working you'll need [nsrllookup](https://github.com/rjhansen/nsrllookup)

To run nsrllookup against your instance of nsrlsvr:

```bash
echo $MD5 | nsrllookup -s localhost -p 9120 -k
```

Which will return $MD5 if present in NSRLFile.txt and nothing when $MD5 does not present.

## Running the analysis plugin

First run log2timeline to calculate the hashes:

```bash
log2timeline.py --hashers md5 --storage-file timeline.plaso image.raw
```

**Make sure to enable hashers supported by nsrlsvr, which is md5 in this example.**

Next run psort to tag events:

```bash
psort.py --analysis nsrlsvr --nsrlsvr-hash md5 --nsrlsvr-host localhost --nsrlsvr-port 9120 -o null timeline.plaso
```

The last step would be to export a timeline with the tags:

```bash
psort.py -o dynamic --fields datetime,timestamp_desc,source,source_long,message,parser,tag -w timeline.csv timeline.plaso
```