Repository: google/turbinia

Issue:
fsstat Task attribute error
Seeing this in the latest release:
```
AttributeError
'NoneType' object has no attribute 'type_indicator'
Traceback (most recent call last): File "/usr/local/lib/python3.8/dist-packages/turbinia-20220216-py3.8.egg/turbinia/workers/__init__.py", line 1005, in run_wrapper self.result = self.run(evidence, self.result) File "/usr/local/lib/python3.8/dist-packages/turbinia-20220216-py3.8.egg/turbinia/workers/fsstat.py", line 43, in run if evidence.path_spec.type_indicator == 'XFS': AttributeError: 'NoneType' object has no attribute 'type_indicator'
```

Full log from that Task:
```
2022-02-17 01:24:28 [INFO] Starting Task FsstatTask 48b091f178fa4a3e84d8fb97507d455b                                                                                                                       [7/1876]2022-02-17 01:24:28 [DEBUG] Starting pre-processor for evidence GoogleCloudDisk
2022-02-17 01:24:28 [DEBUG] Adding task 48b091f178fa4a3e84d8fb97507d455b into the resource state file for resource <redacted>.                                           2022-02-17 01:24:28 [DEBUG] The resource state file has been successfully updated.
2022-02-17 01:24:28 [INFO] Disk <redacted> already attached!
2022-02-17 01:24:28 [DEBUG] Pre-processing evidence GoogleCloudDisk is complete, and evidence is in state [MOUNTED: False, ATTACHED: True, DECOMPRESSED: False, CONTAINER_MOUNTED: False]
2022-02-17 01:24:28 [DEBUG] Starting pre-processor for evidence DiskPartition
2022-02-17 01:24:28 [WARNING] Could not find path_spec for location /p15                                                                                                                                           
2022-02-17 01:24:28 [INFO] Running command sudo losetup --show --find -r -o 4194304 --sizelimit 130023424 /dev/disk/by-id/<redacted>
2022-02-17 01:24:28 [DEBUG] Pre-processing evidence DiskPartition is complete, and evidence is in state [MOUNTED: False, ATTACHED: True, DECOMPRESSED: False, CONTAINER_MOUNTED: False]
2022-02-17 01:24:28 [DEBUG] Returning cached config from /etc/turbinia/turbinia.conf instead of reloading config
2022-02-17 01:24:28 [DEBUG] Updating Task FsstatTask in Datastore
2022-02-17 01:24:28 [ERROR] FsstatTask Task failed with exception: ['NoneType' object has no attribute 'type_indicator']
2022-02-17 01:24:28 [ERROR] Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/turbinia-20220216-py3.8.egg/turbinia/workers/__init__.py", line 1005, in run_wrapper
    self.result = self.run(evidence, self.result)
  File "/usr/local/lib/python3.8/dist-packages/turbinia-20220216-py3.8.egg/turbinia/workers/fsstat.py", line 43, in run
    if evidence.path_spec.type_indicator == 'XFS':
AttributeError: 'NoneType' object has no attribute 'type_indicator'

2022-02-17 01:24:28 [DEBUG] Returning cached config from /etc/turbinia/turbinia.conf instead of reloading config
2022-02-17 01:24:28 [ERROR] FsstatTask Task failed with exception: ['NoneType' object has no attribute 'type_indicator']
2022-02-17 01:24:28 [INFO] Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/turbinia-20220216-py3.8.egg/turbinia/workers/__init__.py", line 1005, in run_wrapper
    self.result = self.run(evidence, self.result)
  File "/usr/local/lib/python3.8/dist-packages/turbinia-20220216-py3.8.egg/turbinia/workers/fsstat.py", line 43, in run
    if evidence.path_spec.type_indicator == 'XFS':
AttributeError: 'NoneType' object has no attribute 'type_indicator'

2022-02-17 01:24:28 [DEBUG] Checking TurbiniaTaskResult for pickle serializability
2022-02-17 01:24:28 [DEBUG] Checking TurbiniaTaskResult for JSON serializability
2022-02-17 01:24:28 [INFO] Result check: Successful
2022-02-17 01:24:28 [DEBUG] No state_manager initialized, not updating Task info
2022-02-17 01:24:28 [WARNING] Trying last ditch attempt to close result
2022-02-17 01:24:28 [INFO] Trying last ditch attempt to close result
2022-02-17 01:24:28 [INFO] Task Result was auto-closed from task executor on turbinia-worker-7fd8ff76d9-7bvjr likely due to previous failures.  Previous status: [FsstatTask Task failed with exception: ['NoneType
' object has no attribute 'type_indicator']]
2022-02-17 01:24:28 [INFO] Task Result was auto-closed from task executor on turbinia-worker-7fd8ff76d9-7bvjr likely due to previous failures.  Previous status: [FsstatTask Task failed with exception: ['NoneType
' object has no attribute 'type_indicator']]
2022-02-17 01:24:28 [INFO] Starting post-processor for evidence DiskPartition
2022-02-17 01:24:28 [DEBUG] Evidence state: [MOUNTED: False, ATTACHED: True, DECOMPRESSED: False, CONTAINER_MOUNTED: False]
2022-02-17 01:24:28 [INFO] Running: sudo losetup -d /dev/loop0
2022-02-17 01:24:29 [INFO] Running: sudo losetup -a
2022-02-17 01:24:29 [INFO] losetup device [/dev/loop0] deleted.
2022-02-17 01:24:29 [INFO] Starting post-processor for evidence GoogleCloudDisk
2022-02-17 01:24:29 [DEBUG] Evidence state: [MOUNTED: False, ATTACHED: True, DECOMPRESSED: False, CONTAINER_MOUNTED: False]
2022-02-17 01:24:29 [DEBUG] Last task 48b091f178fa4a3e84d8fb97507d455b remaining. Removing resource <redacted> from the resource state file.
2022-02-17 01:24:29 [DEBUG] The resource state file has been successfully updated.
2022-02-17 01:24:29 [DEBUG] Returning cached config from /etc/turbinia/turbinia.conf instead of reloading config
2022-02-17 01:24:30 [INFO] Detaching disk <redacted> from instance gke-turbinia-prod-cluste-default-pool-eb936c58-lv66
2022-02-17 01:24:36 [INFO] Block device /dev/disk/by-id/<redacted> is no longer attached
2022-02-17 01:24:36 [DEBUG] Not copying source file /var/tmp/b82801fb483744bfa65a4b90fa1724ab/1645061068-48b091f178fa4a3e84d8fb97507d455b-FsstatTask/worker-log.txt already in output dir /var/tmp/b82801fb483744bf
a65a4b90fa1724ab/1645061068-48b091f178fa4a3e84d8fb97507d455b-FsstatTask
2022-02-17 01:24:36 [INFO] Writing /var/tmp/b82801fb483744bfa65a4b90fa1724ab/1645061068-48b091f178fa4a3e84d8fb97507d455b-FsstatTask/worker-log.txt to GCS path output/b82801fb483744bfa65a4b90fa1724ab/1645061068-4
8b091f178fa4a3e84d8fb97507d455b-FsstatTask/worker-log.txt
```


Discussion:
@dfjxs Any chance you could take a look at this?
Sure, will take a look.

Resolving patch:
diff --git a/turbinia/evidence.py b/turbinia/evidence.py
--- a/turbinia/evidence.py
+++ b/turbinia/evidence.py
@@ -578,13 +578,14 @@ class DiskPartition(RawDisk):
     partition_offset (int): Offset of the partition in bytes.
     partition_size (int): Size of the partition in bytes.
     path_spec (dfvfs.PathSpec): Partition path spec.
+    type_indicator (str): Partition type indicator from dfVFS.
   """
 
   POSSIBLE_STATES = [EvidenceState.ATTACHED, EvidenceState.MOUNTED]
 
   def __init__(
       self, partition_location=None, partition_offset=None, partition_size=None,
-      lv_uuid=None, path_spec=None, *args, **kwargs):
+      lv_uuid=None, path_spec=None, type_indicator=None, *args, **kwargs):
     """Initialization for raw volume evidence object."""
 
     self.partition_location = partition_location
@@ -592,6 +593,7 @@ def __init__(
     self.partition_size = partition_size
     self.lv_uuid = lv_uuid
     self.path_spec = path_spec
+    self.type_indicator = type_indicator
     super(DiskPartition, self).__init__(*args, **kwargs)
 
     # This Evidence needs to have a parent
@@ -612,6 +614,7 @@ def _preprocess(self, _, required_states):
         path_specs, self.partition_location)
     if path_spec:
       self.path_spec = path_spec
+      self.type_indicator = path_spec.type_indicator
 
     # In attaching a partition, we create a new loopback device using the
     # partition offset and size.
diff --git a/turbinia/workers/fsstat.py b/turbinia/workers/fsstat.py
--- a/turbinia/workers/fsstat.py
+++ b/turbinia/workers/fsstat.py
@@ -17,13 +17,13 @@
 
 import os
 
-from turbinia import TurbiniaException
 from turbinia.workers import TurbiniaTask
 from turbinia.evidence import EvidenceState as state
 from turbinia.evidence import ReportText
 
 
 class FsstatTask(TurbiniaTask):
+  """Task to run fsstat on an evidence object."""
 
   REQUIRED_STATES = [state.ATTACHED]
 
@@ -38,9 +38,13 @@ def run(self, evidence, result):
         TurbiniaTaskResult object.
     """
     fsstat_output = os.path.join(self.output_dir, 'fsstat.txt')
+
     # Since fsstat does not support XFS, we won't run it when we know the
     # partition is XFS.
-    if evidence.path_spec.type_indicator == 'XFS':
+
+    # Note: an evidence object that was serialized will not have a path_spec
+    # because it is removed prior to JSON serialization in evidence.py:243
+    if evidence.type_indicator == "XFS":
       message = 'Not running fsstat since partition is XFS'
       result.log(message)
       result.close(self, success=True, status=message)
diff --git a/turbinia/workers/partitions.py b/turbinia/workers/partitions.py
--- a/turbinia/workers/partitions.py
+++ b/turbinia/workers/partitions.py
@@ -60,6 +60,7 @@ def _ProcessPartition(self, path_spec):
     partition_offset = None
     partition_size = None
     lv_uuid = None
+    type_indicator = None
 
     # File system location / identifier
     is_lvm = False
@@ -126,7 +127,8 @@ def _ProcessPartition(self, path_spec):
     # Not setting path_spec here as it will need to be generated for each task
     partition_evidence = DiskPartition(
         partition_location=fs_location, partition_offset=partition_offset,
-        partition_size=partition_size, lv_uuid=lv_uuid)
+        partition_size=partition_size, lv_uuid=lv_uuid,
+        type_indicator=type_indicator)
 
     return partition_evidence, status_report
 


Test patch:
diff --git a/turbinia/workers/fsstat_test.py b/turbinia/workers/fsstat_test.py
--- a/turbinia/workers/fsstat_test.py
+++ b/turbinia/workers/fsstat_test.py
@@ -16,9 +16,7 @@
 
 from __future__ import unicode_literals
 
-import os
 import unittest
-import textwrap
 import mock
 
 from turbinia.evidence import ReportText
@@ -41,7 +39,7 @@ def setUp(self):
   def testFsstatRun(self, mock_evidence):
     """Test fsstat task run."""
     self.task.execute = mock.MagicMock(return_value=0)
-    mock_evidence.path_spec.type_indicator = 'EXT'
+    mock_evidence.type_indicator = 'EXT'
     result = self.task.run(mock_evidence, self.result)
 
     # Ensure execute method is being called.
@@ -51,7 +49,7 @@ def testFsstatRun(self, mock_evidence):
 
     # Test for XFS
     self.task.execute.reset_mock()
-    mock_evidence.path_spec.type_indicator = 'XFS'
+    mock_evidence.type_indicator = 'XFS'
     result = self.task.run(mock_evidence, self.result)
 
     # Ensure execute method is not being called.
@@ -59,6 +57,18 @@ def testFsstatRun(self, mock_evidence):
     # Ensure run method returns a TurbiniaTaskResult instance.
     self.assertIsInstance(result, TurbiniaTaskResult)
 
+  @mock.patch('turbinia.evidence.DiskPartition')
+  def testFsstatRunNoPathSpec(self, mock_evidence):
+    """Test fssstat task on an evidence object with no path_spec."""
+    self.task.execute = mock.MagicMock(return_value=0)
+    mock_evidence.path_spec = None
+    result = self.task.run(mock_evidence, self.result)
+
+    # Ensure execute method is not being called.
+    self.task.execute.assert_called_once()
+    # Ensure run method returns a TurbiniaTaskResult instance.
+    self.assertIsInstance(result, TurbiniaTaskResult)
+
 
 if __name__ == '__main__':
   unittest.main()