Repository: log2timeline/dftimewolf

Issue:
Get rid of input / output semantics
They are brittle, and given that we switched to a non-linear execution model, these don't make much sense anymore.

Let's model data exchange between modules using AttributeContainers

Discussion:
codecov[bot]: # [Codecov](https://codecov.io/gh/log2timeline/dftimewolf/pull/280?src=pr&el=h1) Report
> Merging [#280](https://codecov.io/gh/log2timeline/dftimewolf/pull/280?src=pr&el=desc) into [master](https://codecov.io/gh/log2timeline/dftimewolf/commit/0078194d266b36f7bc4532a45893b9230a81b362&el=desc) will **increase** coverage by `4.48%`.
> The diff coverage is `68.99%`.

[![Impacted file tree graph](https://codecov.io/gh/log2timeline/dftimewolf/pull/280/graphs/tree.svg?width=650&height=150&src=pr&token=aNZ3Zz7fzL)](https://codecov.io/gh/log2timeline/dftimewolf/pull/280?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master     #280      +/-   ##
==========================================
+ Coverage   72.00%   76.49%   +4.48%     
==========================================
  Files          26       27       +1     
  Lines        1579     1706     +127     
==========================================
+ Hits         1137     1305     +168     
+ Misses        442      401      -41     
```


| [Impacted Files](https://codecov.io/gh/log2timeline/dftimewolf/pull/280?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [dftimewolf/cli/dftimewolf\_recipes.py](https://codecov.io/gh/log2timeline/dftimewolf/pull/280/diff?src=pr&el=tree#diff-ZGZ0aW1ld29sZi9jbGkvZGZ0aW1ld29sZl9yZWNpcGVzLnB5) | `0.00% <0.00%> (ø)` | |
| [...ftimewolf/lib/processors/gcp\_logging\_timesketch.py](https://codecov.io/gh/log2timeline/dftimewolf/pull/280/diff?src=pr&el=tree#diff-ZGZ0aW1ld29sZi9saWIvcHJvY2Vzc29ycy9nY3BfbG9nZ2luZ190aW1lc2tldGNoLnB5) | `72.65% <0.00%> (ø)` | |
| [dftimewolf/lib/exporters/timesketch.py](https://codecov.io/gh/log2timeline/dftimewolf/pull/280/diff?src=pr&el=tree#diff-ZGZ0aW1ld29sZi9saWIvZXhwb3J0ZXJzL3RpbWVza2V0Y2gucHk=) | `48.31% <35.38%> (+14.98%)` | :arrow_up: |
| [dftimewolf/lib/timesketch\_utils.py](https://codecov.io/gh/log2timeline/dftimewolf/pull/280/diff?src=pr&el=tree#diff-ZGZ0aW1ld29sZi9saWIvdGltZXNrZXRjaF91dGlscy5weQ==) | `55.00% <52.63%> (+12.14%)` | :arrow_up: |
| [dftimewolf/lib/processors/localplaso.py](https://codecov.io/gh/log2timeline/dftimewolf/pull/280/diff?src=pr&el=tree#diff-ZGZ0aW1ld29sZi9saWIvcHJvY2Vzc29ycy9sb2NhbHBsYXNvLnB5) | `80.00% <54.54%> (+41.36%)` | :arrow_up: |
| [dftimewolf/lib/exporters/local\_filesystem.py](https://codecov.io/gh/log2timeline/dftimewolf/pull/280/diff?src=pr&el=tree#diff-ZGZ0aW1ld29sZi9saWIvZXhwb3J0ZXJzL2xvY2FsX2ZpbGVzeXN0ZW0ucHk=) | `86.11% <73.68%> (-13.89%)` | :arrow_down: |
| [dftimewolf/lib/collectors/grr\_hunt.py](https://codecov.io/gh/log2timeline/dftimewolf/pull/280/diff?src=pr&el=tree#diff-ZGZ0aW1ld29sZi9saWIvY29sbGVjdG9ycy9ncnJfaHVudC5weQ==) | `89.76% <75.00%> (-0.79%)` | :arrow_down: |
| [dftimewolf/lib/processors/turbinia.py](https://codecov.io/gh/log2timeline/dftimewolf/pull/280/diff?src=pr&el=tree#diff-ZGZ0aW1ld29sZi9saWIvcHJvY2Vzc29ycy90dXJiaW5pYS5weQ==) | `83.33% <77.77%> (-0.27%)` | :arrow_down: |
| [dftimewolf/lib/collectors/grr\_hosts.py](https://codecov.io/gh/log2timeline/dftimewolf/pull/280/diff?src=pr&el=tree#diff-ZGZ0aW1ld29sZi9saWIvY29sbGVjdG9ycy9ncnJfaG9zdHMucHk=) | `91.69% <78.84%> (-3.57%)` | :arrow_down: |
| [dftimewolf/lib/exporters/gce\_disk\_export.py](https://codecov.io/gh/log2timeline/dftimewolf/pull/280/diff?src=pr&el=tree#diff-ZGZ0aW1ld29sZi9saWIvZXhwb3J0ZXJzL2djZV9kaXNrX2V4cG9ydC5weQ==) | `94.11% <94.11%> (ø)` | |
| ... and [8 more](https://codecov.io/gh/log2timeline/dftimewolf/pull/280/diff?src=pr&el=tree-more) | |

------

[Continue to review full report at Codecov](https://codecov.io/gh/log2timeline/dftimewolf/pull/280?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/log2timeline/dftimewolf/pull/280?src=pr&el=footer). Last update [0078194...21c7e5a](https://codecov.io/gh/log2timeline/dftimewolf/pull/280?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).


Patch:
diff --git a/dftimewolf/lib/collectors/filesystem.py b/dftimewolf/lib/collectors/filesystem.py
index f0ce2a1..c461f31 100644
--- a/dftimewolf/lib/collectors/filesystem.py
+++ b/dftimewolf/lib/collectors/filesystem.py
@@ -5,7 +5,7 @@ import os
 
 from dftimewolf.lib import module
 from dftimewolf.lib.modules import manager as modules_manager
-
+from dftimewolf.lib.containers import containers
 
 class FilesystemCollector(module.BaseModule):
   """Local file system collector.
@@ -39,14 +39,18 @@ class FilesystemCollector(module.BaseModule):
 
   def Process(self):
     """Collects paths from the local file system."""
+    file_containers = []
     for path in self._paths:
       if os.path.exists(path):
-        self.state.output.append((os.path.basename(path), path))
+        container = containers.File(os.path.basename(path), path)
+        file_containers.append(container)
       else:
         self.state.AddError(
             'Path {0:s} does not exist'.format(str(path)), critical=False)
-    if not self.state.output:
+    if not file_containers:
       self.state.AddError('No valid paths collected, bailing', critical=True)
+    for container in file_containers:
+      self.state.StoreContainer(container)
 
 
 modules_manager.ModulesManager.RegisterModule(FilesystemCollector)
diff --git a/dftimewolf/lib/collectors/gcloud.py b/dftimewolf/lib/collectors/gcloud.py
index 4b0b6a5..5d2397a 100644
--- a/dftimewolf/lib/collectors/gcloud.py
+++ b/dftimewolf/lib/collectors/gcloud.py
@@ -66,7 +66,12 @@ class GoogleCloudCollector(module.BaseModule):
           disk.name, new_disk.name))
       new_disk.AddLabels(self._gcp_label)
       self.analysis_vm.AttachDisk(new_disk)
-      self.state.output.append((self.analysis_vm.name, new_disk))
+
+      container = containers.ForensicsVM(
+          name=self.analysis_vm.name,
+          evidence_disk=new_disk,
+          platform='gcp')
+      self.state.StoreContainer(container)
 
   # pylint: disable=arguments-differ,too-many-arguments
   def SetUp(self,
diff --git a/dftimewolf/lib/collectors/grr_hosts.py b/dftimewolf/lib/collectors/grr_hosts.py
index d81079f..d5ce27f 100644
--- a/dftimewolf/lib/collectors/grr_hosts.py
+++ b/dftimewolf/lib/collectors/grr_hosts.py
@@ -4,15 +4,15 @@
 import datetime
 import os
 import re
-import time
 import threading
+import time
 import zipfile
 
 from grr_api_client import errors as grr_errors
-from grr_response_proto import flows_pb2
-from grr_response_proto import timeline_pb2
+from grr_response_proto import flows_pb2, timeline_pb2
 
 from dftimewolf.lib.collectors.grr_base import GRRBaseModule
+from dftimewolf.lib.containers import containers
 from dftimewolf.lib.errors import DFTimewolfError
 from dftimewolf.lib.modules import manager as modules_manager
 
@@ -337,10 +337,14 @@ class GRRArtifactCollector(GRRFlow):
       return
     self._AwaitFlow(client, flow_id)
     collected_flow_data = self._DownloadFiles(client, flow_id)
+
     if collected_flow_data:
       print('{0!s}: Downloaded: {1:s}'.format(flow_id, collected_flow_data))
-      fqdn = client.data.os_info.fqdn.lower()
-      self.state.output.append((fqdn, collected_flow_data))
+      container = containers.File(
+          name=client.data.os_info.fqdn.lower(),
+          path=collected_flow_data
+      )
+      self.state.StoreContainer(container)
 
   def Process(self):
     """Collects artifacts from a host with GRR.
@@ -441,8 +445,11 @@ class GRRFileCollector(GRRFlow):
     collected_flow_data = self._DownloadFiles(client, flow_id)
     if collected_flow_data:
       print('{0!s}: Downloaded: {1:s}'.format(flow_id, collected_flow_data))
-      fqdn = client.data.os_info.fqdn.lower()
-      self.state.output.append((fqdn, collected_flow_data))
+      container = containers.File(
+          name=client.data.os_info.fqdn.lower(),
+          path=collected_flow_data
+      )
+      self.state.StoreContainer(container)
 
   def Process(self):
     """Collects files from a host with GRR.
@@ -511,8 +518,11 @@ class GRRFlowCollector(GRRFlow):
     if collected_flow_data:
       print('{0:s}: Downloaded: {1:s}'.format(
           self.flow_id, collected_flow_data))
-      fqdn = client.data.os_info.fqdn.lower()
-      self.state.output.append((fqdn, collected_flow_data))
+      container = containers.File(
+          name=client.data.os_info.fqdn.lower(),
+          path=collected_flow_data
+      )
+      self.state.StoreContainer(container)
 
 
 class GRRTimelineCollector(GRRFlow):
@@ -580,8 +590,11 @@ class GRRTimelineCollector(GRRFlow):
     collected_flow_data = self._DownloadTimeline(client, flow_id)
     if collected_flow_data:
       print('{0!s}: Downloaded: {1:s}'.format(flow_id, collected_flow_data))
-      fqdn = client.data.os_info.fqdn.lower()
-      self.state.output.append((fqdn, collected_flow_data))
+      container = containers.File(
+          name=client.data.os_info.fqdn.lower(),
+          path=collected_flow_data
+      )
+      self.state.StoreContainer(container)
 
   def Process(self):
     """Collects a timeline from a host with GRR.
diff --git a/dftimewolf/lib/collectors/grr_hunt.py b/dftimewolf/lib/collectors/grr_hunt.py
index c234d4d..18bcd22 100644
--- a/dftimewolf/lib/collectors/grr_hunt.py
+++ b/dftimewolf/lib/collectors/grr_hunt.py
@@ -4,11 +4,12 @@
 import os
 import tempfile
 import zipfile
-import yaml
 
+import yaml
 from grr_response_proto import flows_pb2 as grr_flows
 
 from dftimewolf.lib.collectors import grr_base
+from dftimewolf.lib.containers import containers
 from dftimewolf.lib.modules import manager as modules_manager
 
 
@@ -358,7 +359,9 @@ class GRRHuntDownloader(GRRHunt):
       RuntimeError: if no items specified for collection.
     """
     hunt = self.grr_api.Hunt(self.hunt_id).Get()
-    self.state.output = self._CollectHuntResults(hunt)
+    for description, path in self._CollectHuntResults(hunt):
+      container = containers.File(name=description, path=path)
+      self.state.StoreContainer(container)
 
 
 modules_manager.ModulesManager.RegisterModules([
diff --git a/dftimewolf/lib/containers/containers.py b/dftimewolf/lib/containers/containers.py
index 2bdd9a7..4b67125 100644
--- a/dftimewolf/lib/containers/containers.py
+++ b/dftimewolf/lib/containers/containers.py
@@ -111,3 +111,56 @@ class TicketAttribute(interface.AttributeContainer):
     self.type = type_
     self.name = name
     self.value = value
+
+
+class File(interface.AttributeContainer):
+  """Attribute container definition for generic files.
+
+  Attributes:
+    name (str): Human-friendly name or description of the file.
+    path (str): Full path to the file.
+  """
+  CONTAINER_TYPE = 'file'
+
+  def __init__(self, name, path):
+    """Initializes the attribute.
+
+    Args:
+      name (str): Human-friendly name or description of the file.
+      path (str): Full path to the file.
+    """
+    super(File, self).__init__()
+    self.name = name
+    self.path = path
+
+class ForensicsVM(interface.AttributeContainer):
+  """Attribute container definition for a forensics virtual machine.
+
+  Attributes:
+    name (str): Identifying name for the virtual machine
+    evidence_disk (libcloudforensics.GoogleComputeDisk): The disk containing
+        the forensic evidence. Full definition in
+        libcloudforensics.providers.gcp.internal.GoogleComputeDisk
+    platform (str): The cloud platform where the VM is located. One of
+        {gcp,aws,azure}.
+  """
+  CONTAINER_TYPE = 'forensics_vm'
+
+  def __init__(self, name, evidence_disk, platform):
+    super(ForensicsVM, self).__init__()
+    self.name = name
+    self.evidence_disk = evidence_disk
+    self.platform = platform
+
+
+class URL(interface.AttributeContainer):
+  """Attribute container definition for a Uniform Resource Locator.
+
+  Attributes:
+    path (str): The full path to the URL.
+  """
+  CONTAINER_TYPE = 'url'
+
+  def __init__(self, path):
+    super(URL, self).__init__()
+    self.path = path
diff --git a/dftimewolf/lib/exporters/gce_disk_export.py b/dftimewolf/lib/exporters/gce_disk_export.py
index da30021..84ec51f 100644
--- a/dftimewolf/lib/exporters/gce_disk_export.py
+++ b/dftimewolf/lib/exporters/gce_disk_export.py
@@ -1,12 +1,11 @@
 # -*- coding: utf-8 -*-
 """Export disk image from a GCP project to Google Cloud Storage."""
 
-from __future__ import print_function
-from __future__ import unicode_literals
 
 import os
 from libcloudforensics.providers.gcp.internal import project as gcp_project
 from dftimewolf.lib import module
+from dftimewolf.lib.containers import containers
 from dftimewolf.lib.modules import manager as modules_manager
 
 
@@ -51,11 +50,12 @@ class GoogleCloudDiskExport(module.BaseModule):
     image_object.ExportImage(
         self.gcs_output_location, output_name=self.exported_disk_name)
     image_object.Delete()
-    output_uri = os.path.join(
+    output_url = os.path.join(
         self.gcs_output_location, '{0:s}.tar.gz'.format(
             self.exported_disk_name))
-    print('Disk was exported to: {0:s}'.format(output_uri))
-    self.state.output.append(output_uri)
+    print('Disk was exported to: {0:s}'.format(output_url))
+    container = containers.URL(path=output_url)
+    self.state.StoreContainer(container)
 
   # pylint: disable=arguments-differ
   def SetUp(self,
diff --git a/dftimewolf/lib/exporters/local_filesystem.py b/dftimewolf/lib/exporters/local_filesystem.py
index 0192bc6..22848ea 100644
--- a/dftimewolf/lib/exporters/local_filesystem.py
+++ b/dftimewolf/lib/exporters/local_filesystem.py
@@ -6,6 +6,7 @@ import shutil
 import tempfile
 
 from dftimewolf.lib import module
+from dftimewolf.lib.containers import containers
 from dftimewolf.lib.modules import manager as modules_manager
 
 
@@ -28,32 +29,46 @@ class LocalFilesystemCopy(module.BaseModule):
       target_directory (Optional[str]): path of the directory in which
           collected files will be copied.
     """
-    self._target_directory = target_directory
     if not target_directory:
-      self._target_directory = tempfile.mkdtemp()
-    elif not os.path.exists(target_directory):
-      os.makedirs(target_directory)
+      target_directory = tempfile.mkdtemp(prefix='dftimewolf_local_fs')
+    elif os.path.exists(target_directory):
+      target_directory = os.path.join(target_directory, 'dftimewolf')
+    self._target_directory = target_directory
 
   def Process(self):
     """Checks whether the paths exists and updates the state accordingly."""
-    for _, path in self.state.input:
-      self._CopyFileOrDirectory(path, self._target_directory)
-      print('{0:s} -> {1:s}'.format(path, self._target_directory))
+    for file_container in self.state.GetContainers(containers.File):
+      try:
+        self._CopyFileOrDirectory(file_container.path, self._target_directory)
+      except OSError as exception:
+        self.state.AddError(
+            'Could not copy files to {0:s}: {1!s}'.format(
+                self._target_directory, exception),
+            critical=True)
+        return
+      print('{0:s} -> {1:s}'.format(file_container.path,
+                                    self._target_directory))
 
   def _CopyFileOrDirectory(self, source, destination_directory):
     """Recursively copies files from source to destination_directory.
 
+    Files will be copied to `destination_directory`'s root. Directories
+    will be copied to subdirectories in `destination_directory`.
+
     Args:
       source (str): source file or directory to copy into the destination
           directory.
       destination_directory (str): destination directory in which to copy
           source.
     """
+    counter = 0
     if os.path.isdir(source):
-      for item in os.listdir(source):
-        full_source = os.path.join(source, item)
-        full_destination = os.path.join(destination_directory, item)
-        shutil.copytree(full_source, full_destination)
+      try:
+        shutil.copytree(source, destination_directory)
+      except FileExistsError:
+        new_directory = os.path.join(destination_directory, str(counter))
+        shutil.copytree(source, new_directory)
+        counter += 1
     else:
       shutil.copy2(source, destination_directory)
 
diff --git a/dftimewolf/lib/exporters/timesketch.py b/dftimewolf/lib/exporters/timesketch.py
index 15e97b1..c6de224 100644
--- a/dftimewolf/lib/exporters/timesketch.py
+++ b/dftimewolf/lib/exporters/timesketch.py
@@ -43,13 +43,13 @@ class TimesketchExporter(module.BaseModule):
     Args:
       incident_id (Optional[str]): Incident ID or reference. Used in sketch
           description.
-      sketch_id (Optional[int]): Sketch ID to add the resulting timeline to.
+      sketch_id (Optional[str]): Sketch ID to add the resulting timeline to.
           If not provided, a new sketch is created.
       analyzers (Optional[List[str]): If provided a list of analyzer names
           to run on the sketch after they've been imported to Timesketch.
     """
     self.timesketch_api = timesketch_utils.GetApiClient(self.state)
-    self.incident_id = None
+    self.incident_id = incident_id
     self.sketch_id = int(sketch_id) if sketch_id else None
     sketch = None
 
@@ -108,7 +108,8 @@ class TimesketchExporter(module.BaseModule):
 
     recipe_name = self.state.recipe.get('name', 'no_recipe')
     input_names = []
-    for description, _ in self.state.input:
+    for file_container in self.state.GetContainers(containers.File):
+      description = file_container.name
       if not description:
         continue
       name = description.rpartition('.')[0]
@@ -125,14 +126,19 @@ class TimesketchExporter(module.BaseModule):
       streamer.set_sketch(sketch)
       streamer.set_timeline_name(timeline_name)
 
-      for _, path in self.state.input:
+      for file_container in self.state.GetContainers(containers.File):
+        path = file_container.path
         streamer.add_file(path)
 
     api_root = sketch.api.api_root
     host_url = api_root.partition('api/v1')[0]
     sketch_url = '{0:s}sketches/{1:d}/'.format(host_url, sketch.id)
-    print('Your Timesketch URL is: {0:s}'.format(sketch_url))
-    self.state.output = sketch_url
+    message = 'Your Timesketch URL is: {0:s}'.format(sketch_url)
+    container = containers.Report(
+        module_name='TimesketchExporter',
+        text=message,
+        text_format='markdown')
+    self.state.StoreContainer(container)
 
     for analyzer in self._analyzers:
       results = sketch.run_analyzer(
diff --git a/dftimewolf/lib/processors/gcp_logging_timesketch.py b/dftimewolf/lib/processors/gcp_logging_timesketch.py
index 6d06356..48cec16 100644
--- a/dftimewolf/lib/processors/gcp_logging_timesketch.py
+++ b/dftimewolf/lib/processors/gcp_logging_timesketch.py
@@ -227,7 +227,8 @@ class GCPLoggingTimesketch(BaseModule):
     timeline_name = 'GCP logs {0:s} "{1:s}"'.format(
         logs_container.project_name, logs_container.filter_expression)
 
-    self.state.output.append([timeline_name, output_path])
+    container = containers.File(name=timeline_name, path=output_path)
+    self.state.StoreContainer(container)
 
   def Process(self):
     """Processes GCP logs containers for insertion into Timesketch."""
diff --git a/dftimewolf/lib/processors/grepper.py b/dftimewolf/lib/processors/grepper.py
index e720a8f..7fea483 100644
--- a/dftimewolf/lib/processors/grepper.py
+++ b/dftimewolf/lib/processors/grepper.py
@@ -10,7 +10,7 @@ import PyPDF2
 
 from dftimewolf.lib import module
 from dftimewolf.lib.modules import manager as modules_manager
-
+from dftimewolf.lib.containers import containers
 
 class GrepperSearch(module.BaseModule):
   """Processes a list of file paths with to search for
@@ -37,14 +37,15 @@ class GrepperSearch(module.BaseModule):
 
   def Process(self):
     """Executes grep on the module input."""
-    for _, path in self.state.input:
+    for file_container in self.state.GetContainers(containers.File):
+      path = file_container.path
       log_file_path = os.path.join(self._output_path, 'grepper.log')
       print('Log file: {0:s}'.format(log_file_path))
 
       print('Walking through dir (absolute) = ' + os.path.abspath(path))
       try:
         for root, _, files in os.walk(path):
-          for filename in files:
+          for filename in sorted(files):
             found = set()
             fullpath = '{0:s}/{1:s}'.format(os.path.abspath(root), filename)
             if mimetypes.guess_type(filename)[0] == 'application/pdf':
@@ -56,7 +57,7 @@ class GrepperSearch(module.BaseModule):
                       self._keywords, line, re.IGNORECASE)))
             if [item for item in found if item]:
               output = '{0:s}/{1:s}:{2:s}'.format(path, filename, ','.join(
-                  filter(None, found)))
+                  filter(None, sorted(found))))
               if self._final_output:
                 self._final_output += '\n' + output
               else:
diff --git a/dftimewolf/lib/processors/localplaso.py b/dftimewolf/lib/processors/localplaso.py
index 50da85b..5fb5930 100644
--- a/dftimewolf/lib/processors/localplaso.py
+++ b/dftimewolf/lib/processors/localplaso.py
@@ -7,6 +7,7 @@ import uuid
 
 from dftimewolf.lib import module
 from dftimewolf.lib.modules import manager as modules_manager
+from dftimewolf.lib.containers import containers
 
 
 class LocalPlasoProcessor(module.BaseModule):
@@ -32,7 +33,9 @@ class LocalPlasoProcessor(module.BaseModule):
 
   def Process(self):
     """Executes log2timeline.py on the module input."""
-    for description, path in self.state.input:
+    for file_container in self.state.GetContainers(containers.File, pop=True):
+      description = file_container.name
+      path = file_container.path
       log_file_path = os.path.join(self._output_path, 'plaso.log')
       print('Log file: {0:s}'.format(log_file_path))
 
@@ -64,17 +67,22 @@ class LocalPlasoProcessor(module.BaseModule):
             cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
         _, error = l2t_proc.communicate()
         l2t_status = l2t_proc.wait()
-        if l2t_status:
-          # self.console_out.StdErr(errors)
-          message = ('The log2timeline command {0:s} failed: {1!s}.'
-                     ' Check log file for details.').format(full_cmd, error)
-          self.state.AddError(message, critical=True)
-        self.state.output.append((description, plaso_storage_file_path))
       except OSError as exception:
         self.state.AddError(str(exception), critical=True)
+        return
       # Catch all remaining errors since we want to gracefully report them
       except Exception as exception:  # pylint: disable=broad-except
         self.state.AddError(str(exception), critical=True)
+        return
+
+      if l2t_status:
+        message = ('The log2timeline command {0:s} failed: {1!s}.'
+                   ' Check log file for details.').format(full_cmd, error)
+        self.state.AddError(message, critical=True)
+        return
+
+      container = containers.File(description, plaso_storage_file_path)
+      self.state.StoreContainer(container)
 
 
 modules_manager.ModulesManager.RegisterModule(LocalPlasoProcessor)
diff --git a/dftimewolf/lib/processors/turbinia.py b/dftimewolf/lib/processors/turbinia.py
index 0c76bb5..eb249a0 100644
--- a/dftimewolf/lib/processors/turbinia.py
+++ b/dftimewolf/lib/processors/turbinia.py
@@ -5,11 +5,11 @@ import getpass
 import os
 import tempfile
 
+# We import a class to avoid importing the whole turbinia module.
+from turbinia import TurbiniaException
 from turbinia import client as turbinia_client
 from turbinia import config as turbinia_config
-from turbinia import evidence
-from turbinia import output_manager
-from turbinia import TurbiniaException
+from turbinia import evidence, output_manager
 from turbinia.message import TurbiniaRequest
 
 from dftimewolf.lib import module
@@ -161,10 +161,10 @@ class TurbiniaProcessor(module.BaseModule):
     """Process files with Turbinia."""
     log_file_path = os.path.join(self._output_path, 'turbinia.log')
     print('Turbinia log file: {0:s}'.format(log_file_path))
-
-    if self.state.input and not self.disk_name:
-      _, disk = self.state.input[0]
-      self.disk_name = disk.name
+    vm_containers = self.state.GetContainers(containers.ForensicsVM)
+    if vm_containers and not self.disk_name:
+      forensics_vm = vm_containers[0]
+      self.disk_name = forensics_vm.evidence_disk.name
       print('Using disk {0:s} from previous collector'.format(self.disk_name))
 
     evidence_ = evidence.GoogleCloudDisk(
@@ -239,17 +239,17 @@ class TurbiniaProcessor(module.BaseModule):
 
     if not all_local_paths:
       self.state.AddError('No interesting files could be found.', critical=True)
-    self.state.output = all_local_paths
 
-    for _, path in all_local_paths:
+    for description, path in all_local_paths:
       if path.endswith('BinaryExtractorTask.tar.gz'):
-        self.state.StoreContainer(
-            containers.ThreatIntelligence(
-                name='BinaryExtractorResults', indicator=None, path=path))
+        container = containers.ThreatIntelligence(
+            name='BinaryExtractorResults', indicator=None, path=path)
       if path.endswith('hashes.json'):
-        self.state.StoreContainer(
-            containers.ThreatIntelligence(
-                name='ImageExportHashes', indicator=None, path=path))
+        container = containers.ThreatIntelligence(
+            name='ImageExportHashes', indicator=None, path=path)
+      if path.endswith('.plaso'):
+        container = containers.File(name=description, path=path)
+      self.state.StoreContainer(container)
 
 
 modules_manager.ModulesManager.RegisterModule(TurbiniaProcessor)
diff --git a/dftimewolf/lib/state.py b/dftimewolf/lib/state.py
index 401b7c1..27d0443 100644
--- a/dftimewolf/lib/state.py
+++ b/dftimewolf/lib/state.py
@@ -126,18 +126,23 @@ class DFTimewolfState(object):
     with self._state_lock:
       self.store.setdefault(container.CONTAINER_TYPE, []).append(container)
 
-  def GetContainers(self, container_class):
+  def GetContainers(self, container_class, pop=False):
     """Thread-safe method to retrieve data from the state's store.
 
     Args:
       container_class (type): AttributeContainer class used to filter data.
+      pop (Optional[bool]): Whether to remove the containers from the state when they
+          are retrieved.
 
     Returns:
       list[AttributeContainer]: attribute container objects provided in
           the store that correspond to the container type.
     """
     with self._state_lock:
-      return self.store.get(container_class.CONTAINER_TYPE, [])
+      containers = self.store.get(container_class.CONTAINER_TYPE, [])
+      if pop:
+        self.store[container_class.CONTAINER_TYPE] = []
+      return containers
 
   def _SetupModuleThread(self, module_definition):
     """Calls the module's SetUp() function and sets a threading event for it.
diff --git a/setup.py b/setup.py
index 7692d7e..32123c9 100755
--- a/setup.py
+++ b/setup.py
@@ -6,7 +6,6 @@ import glob
 import os
 import sys
 
-
 def ParseRequirements(filename):
   """Parse python requirements.
 


Test patch:
diff --git a/.github/workflows/unittest-pipenv.yml b/.github/workflows/unittest-pipenv.yml
new file mode 100644
index 0000000..2d5e8aa
--- /dev/null
+++ b/.github/workflows/unittest-pipenv.yml
@@ -0,0 +1,28 @@
+name: pipenv unittests
+
+on:
+  pull_request:
+    types: [opened, synchronize, reopened]
+
+jobs:
+  build:
+
+    runs-on: ubuntu-latest
+    strategy:
+      matrix:
+        os: [ubuntu-latest, macos-latest]
+        python-version: [3.7, 3.8]
+
+    steps:
+    - uses: actions/checkout@v2
+    - name: Set up Python ${{ matrix.python-version }}
+      uses: actions/setup-python@v1
+      with:
+        python-version: ${{ matrix.python-version }}
+    - name: Install dependencies
+      run: |
+        pip install pipenv
+        pipenv install -d
+    - name: Test with unittest
+      run: |
+        pipenv run python -m unittest discover -s tests -p '*.py'
diff --git a/.github/workflows/unittest37.yml b/.github/workflows/unittest38.yml
similarity index 81%
rename from .github/workflows/unittest37.yml
rename to .github/workflows/unittest38.yml
index cf5a24e..5e9aaab 100644
--- a/.github/workflows/unittest37.yml
+++ b/.github/workflows/unittest38.yml
@@ -1,4 +1,4 @@
-name: Run tests
+name: pip unittests
 
 on:
   pull_request:
@@ -10,8 +10,8 @@ jobs:
     runs-on: ubuntu-latest
     strategy:
       matrix:
-        os: [ubuntu-latest, macos-latest]
-        python-version: [3.6, 3.7, 3.8]
+        os: [ubuntu-latest]
+        python-version: [3.8]
 
     steps:
     - uses: actions/checkout@v2
@@ -21,7 +21,6 @@ jobs:
         python-version: ${{ matrix.python-version }}
     - name: Install dependencies
       run: |
-        pip install pip==19.3.1
         pip install -r requirements.txt
         pip install -r requirements-dev.txt
     - name: Test with unittest
diff --git a/tests/lib/collectors/filesystem.py b/tests/lib/collectors/filesystem.py
index 012d686..a5e8c13 100644
--- a/tests/lib/collectors/filesystem.py
+++ b/tests/lib/collectors/filesystem.py
@@ -8,6 +8,7 @@ import mock
 
 from dftimewolf.lib import state
 from dftimewolf.lib.collectors import filesystem
+from dftimewolf.lib.containers import containers
 
 from dftimewolf import config
 
@@ -29,11 +30,11 @@ class LocalFileSystemTest(unittest.TestCase):
     filesystem_collector.SetUp(paths=fake_paths)
     mock_exists.return_value = True
     filesystem_collector.Process()
-    expected_output = [
-        ('1', '/fake/path/1'),
-        ('2', '/fake/path/2')
-    ]
-    self.assertEqual(test_state.output, expected_output)
+    files = test_state.GetContainers(containers.File)
+    self.assertEqual(files[0].path, '/fake/path/1')
+    self.assertEqual(files[0].name, '1')
+    self.assertEqual(files[1].path, '/fake/path/2')
+    self.assertEqual(files[1].name, '2')
 
   @mock.patch('dftimewolf.lib.state.DFTimewolfState.AddError')
   def testSetup(self, mock_add_error):
diff --git a/tests/lib/collectors/gcloud.py b/tests/lib/collectors/gcloud.py
index 71c221c..e7ea101 100644
--- a/tests/lib/collectors/gcloud.py
+++ b/tests/lib/collectors/gcloud.py
@@ -10,6 +10,7 @@ from libcloudforensics.providers.gcp.internal import compute
 
 from dftimewolf import config
 from dftimewolf.lib import state
+from dftimewolf.lib.containers import containers
 from dftimewolf.lib.collectors import gcloud
 
 FAKE_PROJECT = gcp_project.GoogleCloudProject(
@@ -137,8 +138,10 @@ class GoogleCloudCollectorTest(unittest.TestCase):
         None,
         FAKE_DISK.zone,
         disk_name=FAKE_DISK.name)
-    self.assertEqual(test_state.output[0][0], 'fake-analysis-vm')
-    self.assertEqual(test_state.output[0][1].name, 'disk1-copy')
+    forensics_vms = test_state.GetContainers(containers.ForensicsVM)
+    forensics_vm = forensics_vms[0]
+    self.assertEqual(forensics_vm.name, 'fake-analysis-vm')
+    self.assertEqual(forensics_vm.evidence_disk.name, 'disk1-copy')
     mock_AddLabels.assert_has_calls([mock.call({'incident_id': 'fake_incident_id'})])
 
   # pylint: disable=line-too-long,invalid-name
diff --git a/tests/lib/collectors/gcloud_e2e.py b/tests/lib/collectors/gcloud_e2e.py
index d92dbb1..22669b2 100644
--- a/tests/lib/collectors/gcloud_e2e.py
+++ b/tests/lib/collectors/gcloud_e2e.py
@@ -14,10 +14,10 @@
 # limitations under the License.
 """End to end test for the Google Cloud Collector."""
 import json
+import logging
 import os
 import time
 import unittest
-import logging
 
 from googleapiclient.errors import HttpError
 from libcloudforensics.providers.gcp.internal import project as gcp_project
@@ -26,6 +26,7 @@ from libcloudforensics.providers.gcp.internal import compute, common
 from dftimewolf import config
 from dftimewolf.lib import state
 from dftimewolf.lib.collectors import gcloud
+from dftimewolf.lib.containers import containers
 
 log = logging.getLogger(__name__)
 
@@ -108,8 +109,10 @@ class EndToEndTest(unittest.TestCase):
 
     # The forensic instance should be live in the analysis GCP project and
     # the disk should be attached
-    analysis_vm_name = self.test_state.output[0][0]
-    expected_disk_name = self.test_state.output[0][1].name
+    forensics_vms = self.test_state.GetContainers(containers.ForensicsVM)
+    analysis_vm = forensics_vms[0]
+    analysis_vm_name = analysis_vm.name
+    expected_disk_name = analysis_vm.evidence_disk.name
 
     gce_instances_client = self.gcp_client.GceApi().instances()
     request = gce_instances_client.get(
@@ -157,8 +160,10 @@ class EndToEndTest(unittest.TestCase):
 
     # The forensic instance should be live in the analysis GCP project and
     # the disk should be attached
-    analysis_vm_name = self.test_state.output[0][0]
-    expected_disk_name = self.test_state.output[0][1].name
+    forensics_vms = self.test_state.GetContainers(containers.ForensicsVM)
+    analysis_vm = forensics_vms[0]
+    analysis_vm_name = analysis_vm.name
+    expected_disk_name = analysis_vm.evidence_disk.name
 
     gce_instances_client = self.gcp_client.GceApi().instances()
     request = gce_instances_client.get(
diff --git a/tests/lib/collectors/grr_hosts.py b/tests/lib/collectors/grr_hosts.py
index 2005707..7d427f0 100644
--- a/tests/lib/collectors/grr_hosts.py
+++ b/tests/lib/collectors/grr_hosts.py
@@ -3,19 +3,19 @@
 """Tests the GRR host collectors."""
 
 import unittest
-import six
-import mock
 
-from grr_response_proto import flows_pb2
+import mock
+import six
 from grr_api_client import errors as grr_errors
+from grr_response_proto import flows_pb2
+from tests.lib.collectors.test_data import mock_grr_hosts
 
 from dftimewolf import config
 from dftimewolf.lib import state
 from dftimewolf.lib.collectors import grr_hosts
+from dftimewolf.lib.containers import containers
 from dftimewolf.lib.errors import DFTimewolfError
 
-from tests.lib.collectors.test_data import mock_grr_hosts
-
 
 # Extensive access to protected members for testing, and mocking of classes.
 # pylint: disable=protected-access,invalid-name
@@ -206,7 +206,6 @@ class GRRArtifactCollectorTest(unittest.TestCase):
     )
     self.grr_artifact_collector.Process()
     kwargs = mock_ArtifactCollectorFlowArgs.call_args[1]
-    # raise ValueError(str(kwargs[1]))
     self.assertFalse(kwargs['apply_parsers'])  # default argument
     self.assertTrue(kwargs['ignore_interpolation_errors'])  # default argument
     self.assertTrue(kwargs['use_tsk'])
@@ -233,9 +232,11 @@ class GRRArtifactCollectorTest(unittest.TestCase):
     mock_DownloadFiles.assert_called_with(
         mock_grr_hosts.MOCK_CLIENT_LIST[1], mock_grr_hosts.MOCK_FLOW.flow_id
     )
-    self.assertEqual(len(self.test_state.output), 1)
-    self.assertEqual(self.test_state.output[0][0], 'tomchop')
-    self.assertEqual(self.test_state.output[0][1], '/tmp/tmpRandom/tomchop')
+    results = self.test_state.GetContainers(containers.File)
+    self.assertEqual(len(results), 1)
+    result = results[0]
+    self.assertEqual(result.name, 'tomchop')
+    self.assertEqual(result.path, '/tmp/tmpRandom/tomchop')
 
 
 class GRRFileCollectorTest(unittest.TestCase):
@@ -285,7 +286,11 @@ class GRRFileCollectorTest(unittest.TestCase):
                 action_type=flows_pb2.FileFinderAction.STAT)
         )
     )
-    self.assertEqual(self.test_state.output[0], ('tomchop', '/tmp/something'))
+    results = self.test_state.GetContainers(containers.File)
+    self.assertEqual(len(results), 1)
+    result = results[0]
+    self.assertEqual(result.name, 'tomchop')
+    self.assertEqual(result.path, '/tmp/something')
 
 
 class GRRFlowCollector(unittest.TestCase):
@@ -317,7 +322,11 @@ class GRRFlowCollector(unittest.TestCase):
     self.grr_flow_collector.Process()
     mock_DownloadFiles.assert_called_once_with(
         mock_grr_hosts.MOCK_CLIENT_RECENT, 'F:12345')
-    self.assertEqual(self.test_state.output[0], ('tomchop', '/tmp/something'))
+    results = self.test_state.GetContainers(containers.File)
+    self.assertEqual(len(results), 1)
+    result = results[0]
+    self.assertEqual(result.name, 'tomchop')
+    self.assertEqual(result.path, '/tmp/something')
 
 
 class GRRTimelineCollector(unittest.TestCase):
@@ -366,7 +375,11 @@ class GRRTimelineCollector(unittest.TestCase):
     self.grr_timeline_collector.Process()
     mock_DownloadTimeline.assert_called_once_with(
         mock_grr_hosts.MOCK_CLIENT_RECENT, 'F:12345')
-    self.assertEqual(self.test_state.output[0], ('tomchop', '/tmp/something'))
+    results = self.test_state.GetContainers(containers.File)
+    self.assertEqual(len(results), 1)
+    result = results[0]
+    self.assertEqual(result.name, 'tomchop')
+    self.assertEqual(result.path, '/tmp/something')
 
 if __name__ == '__main__':
   unittest.main()
diff --git a/tests/lib/exporters/gce_disk_export.py b/tests/lib/exporters/gce_disk_export.py
index 2bada70..ab2e798 100644
--- a/tests/lib/exporters/gce_disk_export.py
+++ b/tests/lib/exporters/gce_disk_export.py
@@ -2,16 +2,16 @@
 # -*- coding: utf-8 -*-
 """Tests the GoogleCloudDiskExport."""
 
-from __future__ import unicode_literals
-
-import unittest
 import os
+import unittest
+
 import mock
 from libcloudforensics.providers.gcp.internal import project as gcp_project
 from libcloudforensics.providers.gcp.internal import compute
 
 from dftimewolf import config
 from dftimewolf.lib import state
+from dftimewolf.lib.containers import containers
 from dftimewolf.lib.exporters import gce_disk_export
 
 
@@ -108,10 +108,11 @@ class GoogleCloudDiskExportTest(unittest.TestCase):
         'gs://fake-bucket', output_name='{0:s}-image-df-export-temp'.format(
             'fake-source-disk'))
     mock_delete_image.assert_called_once()
-    output_uri = os.path.join(
+    output_url = os.path.join(
         'gs://fake-bucket', '{0:s}-image-df-export-temp.tar.gz'.format(
             'fake-source-disk'))
-    self.assertEqual(test_state.output[0], output_uri)
+    urls = test_state.GetContainers(containers.URL)
+    self.assertEqual(urls[0].path, output_url)
 
 
 if __name__ == '__main__':
diff --git a/tests/lib/exporters/local_filesystem.py b/tests/lib/exporters/local_filesystem.py
index 800c1dd..40257cb 100644
--- a/tests/lib/exporters/local_filesystem.py
+++ b/tests/lib/exporters/local_filesystem.py
@@ -3,14 +3,14 @@
 """Tests the local filesystem exporter."""
 
 import unittest
+
 import mock
 
+from dftimewolf import config
 from dftimewolf.lib import state
+from dftimewolf.lib.containers import containers
 from dftimewolf.lib.exporters import local_filesystem
 
-from dftimewolf import config
-
-
 FAKE_PATHS = {
     '/fake/evidence_directory': ['file1', 'file2'],
     '/fake/evidence_file': None
@@ -50,17 +50,18 @@ class LocalFileSystemTest(unittest.TestCase):
                   mock_copytree):
     """Tests that the module processes input correctly."""
     test_state = state.DFTimewolfState(config.Config)
-    test_state.input = [
-        ('description', '/fake/evidence_directory'),
-        ('description2', '/fake/evidence_file'),
-    ]
+    test_state.StoreContainer(containers.File(
+        name='description', path='/fake/evidence_directory'))
+    test_state.StoreContainer(containers.File(
+        name='description2', path='/fake/evidence_file'))
     mock_mkdtemp.return_value = '/fake/random'
+
     local_filesystem_copy = local_filesystem.LocalFilesystemCopy(test_state)
     local_filesystem_copy.SetUp()
     local_filesystem_copy.Process()
+
     mock_copytree.assert_has_calls([
-        mock.call('/fake/evidence_directory/file1', '/fake/random/file1'),
-        mock.call('/fake/evidence_directory/file2', '/fake/random/file2'),
+        mock.call('/fake/evidence_directory', '/fake/random'),
     ])
     mock_copy2.assert_called_with('/fake/evidence_file', '/fake/random')
 
@@ -74,15 +75,19 @@ class LocalFileSystemTest(unittest.TestCase):
     # pylint: disable=protected-access
     self.assertEqual(local_filesystem_copy._target_directory, '/fake/random')
 
-  @mock.patch('os.makedirs')
-  def testSetupError(self, mock_makedirs):
+  @mock.patch('os.path.isdir')
+  @mock.patch('shutil.copytree')
+  def testSetupError(self, mock_copytree, mock_isdir):
     """Tests that an error is generated if target_directory is unavailable."""
-    mock_makedirs.side_effect = OSError('FAKEERROR')
+    mock_copytree.side_effect = OSError('FAKEERROR')
+    mock_isdir.return_value = False
     test_state = state.DFTimewolfState(config.Config)
+    test_state.StoreContainer(
+        containers.File(name='blah', path='/sourcefile'))
     local_filesystem_copy = local_filesystem.LocalFilesystemCopy(test_state)
-    with self.assertRaises(OSError) as exception:
-      local_filesystem_copy.SetUp(target_directory="/nonexistent")
-      self.assertEqual(str(exception), 'FAKEERROR')
+    local_filesystem_copy.SetUp(target_directory="/nonexistent")
+    local_filesystem_copy.Process()
+    self.assertEqual(len(test_state.errors), 1)
 
   @mock.patch('os.makedirs')
   def testSetupManualDir(self, mock_makedirs):
diff --git a/tests/lib/exporters/timesketch.py b/tests/lib/exporters/timesketch.py
index e27048e..816868d 100644
--- a/tests/lib/exporters/timesketch.py
+++ b/tests/lib/exporters/timesketch.py
@@ -4,10 +4,11 @@
 
 import unittest
 
-from dftimewolf.lib import state
-from dftimewolf.lib.exporters import timesketch
+import mock
 
 from dftimewolf import config
+from dftimewolf.lib import state
+from dftimewolf.lib.exporters import timesketch
 
 
 class TimesketchExporterTest(unittest.TestCase):
@@ -19,6 +20,64 @@ class TimesketchExporterTest(unittest.TestCase):
     timesketch_exporter = timesketch.TimesketchExporter(test_state)
     self.assertIsNotNone(timesketch_exporter)
 
+  # pylint: disable=invalid-name
+  @mock.patch('dftimewolf.lib.timesketch_utils.GetApiClient')
+  def testSetup(self, mock_GetApiClient):
+    """Tests the SetUp function."""
+    mock_sketch = mock.Mock()
+    mock_sketch.id = 1234
+    mock_api_client = mock.Mock()
+    mock_api_client.create_sketch.return_value = mock_sketch
+    mock_GetApiClient.return_value = mock_api_client
+    test_state = state.DFTimewolfState(config.Config)
+    timesketch_exporter = timesketch.TimesketchExporter(test_state)
+    timesketch_exporter.SetUp(
+        incident_id=None,
+        sketch_id=None,
+        analyzers=None
+    )
+    self.assertEqual(timesketch_exporter.sketch_id, 1234)
+    mock_api_client.create_sketch.assert_called_with(
+        'Untitled sketch', 'Sketch generated by dfTimewolf')
+
+  # pylint: disable=invalid-name
+  @mock.patch('dftimewolf.lib.timesketch_utils.GetApiClient')
+  def testSetupForceIncidentId(self, mock_GetApiClient):
+    """Tests the SetUp function when an incident ID is passed."""
+    mock_sketch = mock.Mock()
+    mock_sketch.id = 1234
+    mock_api_client = mock.Mock()
+    mock_api_client.create_sketch.return_value = mock_sketch
+    mock_GetApiClient.return_value = mock_api_client
+    test_state = state.DFTimewolfState(config.Config)
+    timesketch_exporter = timesketch.TimesketchExporter(test_state)
+    timesketch_exporter.SetUp(
+        incident_id='9999',
+        sketch_id=None,
+        analyzers=None
+    )
+    self.assertEqual(timesketch_exporter.sketch_id, 1234)
+    mock_api_client.create_sketch.assert_called_with(
+        'Sketch for incident ID: 9999', 'Sketch generated by dfTimewolf')
+
+  # pylint: disable=invalid-name
+  @mock.patch('dftimewolf.lib.timesketch_utils.GetApiClient')
+  def testSetupForceSketchId(self, mock_GetApiClient):
+    """Tests the SetUp function when an incident ID is passed."""
+    mock_sketch = mock.Mock()
+    mock_api_client = mock.Mock()
+    mock_api_client.get_sketch.return_value = mock_sketch
+    mock_GetApiClient.return_value = mock_api_client
+    test_state = state.DFTimewolfState(config.Config)
+    timesketch_exporter = timesketch.TimesketchExporter(test_state)
+    timesketch_exporter.SetUp(
+        incident_id='9999',
+        sketch_id='6666',
+        analyzers=None
+    )
+    self.assertEqual(timesketch_exporter.sketch_id, 6666)
+    mock_api_client.get_sketch.assert_called_with(6666)
+
 
 if __name__ == '__main__':
   unittest.main()
diff --git a/tests/lib/processors/grepper.py b/tests/lib/processors/grepper.py
index cd8a665..b542ab3 100644
--- a/tests/lib/processors/grepper.py
+++ b/tests/lib/processors/grepper.py
@@ -4,16 +4,16 @@
 
 import unittest
 
+from dftimewolf import config
 from dftimewolf.lib import state
+from dftimewolf.lib.containers import containers
 from dftimewolf.lib.processors import grepper
 
-from dftimewolf import config
-
 
 class GrepperTest(unittest.TestCase):
   """Test case for the grep function. """
 
-  def TestSingleGrep(self):
+  def testSingleGrep(self):
     """Test just single keyword grep search on text files."""
     test_state = state.DFTimewolfState(config.Config)
     base_grepper_search = grepper.GrepperSearch(test_state)
@@ -22,16 +22,18 @@ class GrepperTest(unittest.TestCase):
     )
     # Put here a path to a test directory where you have files to grep on the
     # above keyword. This is to simulate the path received an input from GRR
-    base_grepper_search.state.input = \
-      [['Test description', '../collectors/test_data/grepper_test_dir']]
+    test_state.StoreContainer(containers.File(
+        name='Test description',
+        path='tests/lib/collectors/test_data/grepper_test_dir'
+    ))
     base_grepper_search.Process()
     # pylint: disable=protected-access
     self.assertEqual(
         base_grepper_search._keywords, 'foo|lorem|meow|triage|bar|homebrew')
 
+    # pylint: disable=line-too-long
     self.assertEqual(
         base_grepper_search._final_output,
-        '../collectors/test_data/grepper_test_dir/grepper_test.txt:bar,foo,'
-        'lorem,triage\n'
-        '../collectors/test_data/grepper_test_dir/1test.pdf:homebrew\n'
-        '../collectors/test_data/grepper_test_dir/grepper_test2.txt:foo')
+        'tests/lib/collectors/test_data/grepper_test_dir/1test.pdf:homebrew\n'
+        'tests/lib/collectors/test_data/grepper_test_dir/grepper_test.txt:bar,foo,lorem,triage\n'
+        'tests/lib/collectors/test_data/grepper_test_dir/grepper_test2.txt:foo')
diff --git a/tests/lib/processors/localplaso.py b/tests/lib/processors/localplaso.py
index a7790c4..bd07ba8 100644
--- a/tests/lib/processors/localplaso.py
+++ b/tests/lib/processors/localplaso.py
@@ -3,9 +3,11 @@
 """Tests the localplaso processor."""
 
 import unittest
+import mock
 
 from dftimewolf.lib import state
 from dftimewolf.lib.processors import localplaso
+from dftimewolf.lib.containers import containers
 
 from dftimewolf import config
 
@@ -19,6 +21,28 @@ class LocalPlasoTest(unittest.TestCase):
     local_plaso_processor = localplaso.LocalPlasoProcessor(test_state)
     self.assertIsNotNone(local_plaso_processor)
 
+  # pylint: disable=invalid-name
+  @mock.patch('subprocess.Popen')
+  def testProcessing(self, mock_Popen):
+    """Tests that the correct number of containers is added."""
+    test_state = state.DFTimewolfState(config.Config)
+    mock_popen_object = mock.Mock()
+    mock_popen_object.communicate.return_value = (None, None)
+    mock_popen_object.wait.return_value = False
+    mock_Popen.return_value = mock_popen_object
+
+    local_plaso_processor = localplaso.LocalPlasoProcessor(test_state)
+    test_state.StoreContainer(
+        containers.File(name='test', path='/notexist/test'))
+    local_plaso_processor.SetUp()
+    local_plaso_processor.Process()
+    mock_Popen.assert_called_once()
+    args = mock_Popen.call_args[0][0] # Get positional arguments of first call
+    self.assertEqual(args[9], '/notexist/test')
+    plaso_path = args[8] # Dynamically generated path to the plaso file
+    self.assertEqual(
+        test_state.GetContainers(containers.File)[0].path,
+        plaso_path)
 
 if __name__ == '__main__':
   unittest.main()
diff --git a/tests/lib/processors/turbinia.py b/tests/lib/processors/turbinia.py
index 0ad0131..70343c5 100644
--- a/tests/lib/processors/turbinia.py
+++ b/tests/lib/processors/turbinia.py
@@ -14,6 +14,7 @@ from dftimewolf.lib import state
 current_dir = os.path.dirname(os.path.realpath(__file__))
 os.environ['TURBINIA_CONFIG_PATH'] = os.path.join(current_dir, 'test_data')
 # pylint: disable=wrong-import-position
+from dftimewolf.lib.containers import containers
 from dftimewolf.lib.processors import turbinia
 
 from dftimewolf import config
@@ -108,7 +109,6 @@ class TurbiniaProcessorTest(unittest.TestCase):
       self.assertEqual(error_msg, expected_error)
       self.assertEqual(is_critical, True)
 
-  @mock.patch('dftimewolf.lib.state.DFTimewolfState.StoreContainer')
   @mock.patch('os.path.exists')
   @mock.patch('turbinia.output_manager.GCSOutputWriter')
   @mock.patch('turbinia.evidence.GoogleCloudDisk')
@@ -118,8 +118,7 @@ class TurbiniaProcessorTest(unittest.TestCase):
                   _mock_TurbiniaClient,
                   mock_GoogleCloudDisk,
                   mock_GCSOutputWriter,
-                  mock_exists,
-                  mock_StoreContainer):
+                  mock_exists):
     """Tests that the processor processes data correctly."""
 
     test_state = state.DFTimewolfState(config.Config)
@@ -135,6 +134,7 @@ class TurbiniaProcessorTest(unittest.TestCase):
         'saved_paths': [
             '/fake/data.plaso',
             '/fake/data2.plaso',
+            '/another/random/file.txt',
             'gs://BinaryExtractorTask.tar.gz',
         ]
     }]
@@ -167,20 +167,19 @@ class TurbiniaProcessorTest(unittest.TestCase):
         local_output_dir=turbinia_processor._output_path
     )
     self.assertEqual(test_state.errors, [])
-    self.assertEqual(test_state.output, [
-        ('turbinia-project-disk-1', '/fake/data.plaso'),
-        ('turbinia-project-disk-1', '/fake/data2.plaso'),
-        ('turbinia-project-disk-1', '/local/BinaryExtractorTask.tar.gz'),
-    ])
-    self.assertEqual(
-        mock_StoreContainer.call_args[0][0].CONTAINER_TYPE,
-        'threat_intelligence')
-    self.assertEqual(
-        mock_StoreContainer.call_args[0][0].name,
-        'BinaryExtractorResults')
-    self.assertEqual(
-        mock_StoreContainer.call_args[0][0].path,
-        '/local/BinaryExtractorTask.tar.gz')
+    ti_containers = test_state.GetContainers(containers.ThreatIntelligence)
+    file_containers = test_state.GetContainers(containers.File)
+
+    # Make sure that file.txt is ignored
+    self.assertEqual(len(file_containers), 2)
+
+    self.assertEqual(ti_containers[0].name, 'BinaryExtractorResults')
+    self.assertEqual(ti_containers[0].path, '/local/BinaryExtractorTask.tar.gz')
+
+    self.assertEqual(file_containers[0].name, 'turbinia-project-disk-1')
+    self.assertEqual(file_containers[1].name, 'turbinia-project-disk-1')
+    self.assertEqual(file_containers[0].path, '/fake/data.plaso')
+    self.assertEqual(file_containers[1].path, '/fake/data2.plaso')
 
   @mock.patch('turbinia.output_manager.GCSOutputWriter')
   # pylint: disable=invalid-name