Repository: log2timeline/plaso

Issue #1036: Analysis plugin nsrlsvr: Incorrect results 
Created: 2016-10-07T14:08:00Z
Plaso version:
1.5.0 Release
Operating system Plaso is running on:
XUbuntu 64 bit
Installation method:
Installed from GiFT
Description of problem:
Using the nsrlsvr analysis plugin of psort.py yields incorrect results. The response from the server is mis-interpreted, causing the suggestion that no hash is listed at all.
Source data:
Problem seems to be related to line-endings, and may depend on the OS of the client or the OS of the NSRL server. Problem can be fixed by rstrip-ing the response from the server in line 67 of plaso\analysis\nsrlsvr.py :
response = nsrl_socket.recv(self._RECEIVE_BUFFER_SIZE).rstrip()

Note that #1035 is related and needs fixing before this issue can be debugged.

Duplicate issue #1149: IOError: Unable to write data entry size value out of bounds.
Created: 2017-01-24T06:19:32Z
Plaso version:
log2timeline version 1.5.2_20170123
Operating system Plaso is running on:
Ubuntu 16.04.1
Installation method:
PPA:GIFT/Dev, Git clone dev PLASO version.
Install dependencies, run "check_dependencies.py" & "run_tests.py"
Description of problem:
Please describe the problem in as much detail as possible, including relevant command line arguments, flags etc.
What does the tool not do that it should? What would you like it to do?
Stopped processing after sometime running "kitchen sink" mode on an Encase image.
Debug output/tracebacks:
Traceback (most recent call last):
  File "/usr/local/bin/log2timeline.py", line 4, in <module>
    __import__('pkg_resources').run_script('plaso==1.5.2.post20170123', 'log2timeline.py')
  File "/usr/local/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 739, in run_script
    self.require(requires)[0].run_script(script_name, ns)
  File "/usr/local/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 1494, in run_script
    exec(code, namespace, namespace)
  File "/usr/local/lib/python2.7/dist-packages/plaso-1.5.2.post20170123-py2.7.egg/EGG-INFO/scripts/log2timeline.py", line 678, in <module>
    if not Main():
  File "/usr/local/lib/python2.7/dist-packages/plaso-1.5.2.post20170123-py2.7.egg/EGG-INFO/scripts/log2timeline.py", line 664, in Main
    tool.ProcessSources()
  File "/usr/local/lib/python2.7/dist-packages/plaso-1.5.2.post20170123-py2.7.egg/EGG-INFO/scripts/log2timeline.py", line 575, in ProcessSources
    worker_memory_limit=self._worker_memory_limit)
  File "/usr/local/lib/python2.7/dist-packages/plaso-1.5.2.post20170123-py2.7.egg/plaso/frontend/extraction_frontend.py", line 391, in ProcessSources
    worker_memory_limit=worker_memory_limit)
  File "/usr/local/lib/python2.7/dist-packages/plaso-1.5.2.post20170123-py2.7.egg/plaso/multi_processing/task_engine.py", line 770, in ProcessSources
    filter_find_specs=filter_find_specs)
  File "/usr/local/lib/python2.7/dist-packages/plaso-1.5.2.post20170123-py2.7.egg/plaso/multi_processing/task_engine.py", line 275, in _ProcessSources
    self._ScheduleTasks(storage_writer)
  File "/usr/local/lib/python2.7/dist-packages/plaso-1.5.2.post20170123-py2.7.egg/plaso/multi_processing/task_engine.py", line 407, in _ScheduleTasks
    self._MergeTaskStorage(storage_writer)
  File "/usr/local/lib/python2.7/dist-packages/plaso-1.5.2.post20170123-py2.7.egg/plaso/multi_processing/task_engine.py", line 205, in _MergeTaskStorage
    maximum_number_of_containers=self._MAXIMUM_NUMBER_OF_CONTAINERS)
  File "/usr/local/lib/python2.7/dist-packages/plaso-1.5.2.post20170123-py2.7.egg/plaso/storage/gzip_file.py", line 415, in MergeAttributeContainers
    self._AddAttributeContainer(attribute_container)
  File "/usr/local/lib/python2.7/dist-packages/plaso-1.5.2.post20170123-py2.7.egg/plaso/storage/gzip_file.py", line 348, in _AddAttributeContainer
    self._storage_writer.AddEvent(attribute_container)
  File "/usr/local/lib/python2.7/dist-packages/plaso-1.5.2.post20170123-py2.7.egg/plaso/storage/zip_file.py", line 3050, in AddEvent
    self._storage_file.AddEvent(event)
  File "/usr/local/lib/python2.7/dist-packages/plaso-1.5.2.post20170123-py2.7.egg/plaso/storage/zip_file.py", line 2346, in AddEvent
    self._WriteSerializedEvents()
  File "/usr/local/lib/python2.7/dist-packages/plaso-1.5.2.post20170123-py2.7.egg/plaso/storage/zip_file.py", line 2021, in _WriteSerializedEvents
    self._serialized_events_heap, self._event_stream_number)
  File "/usr/local/lib/python2.7/dist-packages/plaso-1.5.2.post20170123-py2.7.egg/plaso/storage/zip_file.py", line 2055, in _WriteSerializedEventsHeap
    entry_data_offset = data_stream.WriteEntry(entry_data)
  File "/usr/local/lib/python2.7/dist-packages/plaso-1.5.2.post20170123-py2.7.egg/plaso/storage/zip_file.py", line 565, in WriteEntry
    raise IOError(u'Unable to write data entry size value out of bounds.')
IOError: Unable to write data entry size value out of bounds.
2017-01-24 04:33:12,385 [WARNING] (MainProcess) PID:3458 <zeromq_queue> main_task_queue timeout waiting for a request.
2017-01-24 04:33:21,953 [WARNING] (MainProcess) PID:3458 <zeromq_queue> main_task_queue timeout waiting for a request.
2017-01-24 04:33:31,410 [WARNING] (MainProcess) PID:3458 <zeromq_queue> main_task_queue timeout waiting for a request.
2017-01-24 04:33:41,243 [WARNING] (MainProcess) PID:3458 <zeromq_queue> main_task_queue timeout waiting for a request.
2017-01-24 04:33:51,040 [WARNING] (MainProcess) PID:3458 <zeromq_queue> main_task_queue timeout waiting for a request.

Another run with --disable_zeromq has the following error:
Traceback (most recent call last):
  File "/usr/local/bin/log2timeline.py", line 4, in <module>
    __import__('pkg_resources').run_script('plaso==1.5.2.post20170123', 'log2timeline.py')
  File "/usr/local/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 739, in run_script
    self.require(requires)[0].run_script(script_name, ns)
  File "/usr/local/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 1494, in run_script
    exec(code, namespace, namespace)
  File "/usr/local/lib/python2.7/dist-packages/plaso-1.5.2.post20170123-py2.7.egg/EGG-INFO/scripts/log2timeline.py", line 678, in <module>
    if not Main():
  File "/usr/local/lib/python2.7/dist-packages/plaso-1.5.2.post20170123-py2.7.egg/EGG-INFO/scripts/log2timeline.py", line 664, in Main
    tool.ProcessSources()
  File "/usr/local/lib/python2.7/dist-packages/plaso-1.5.2.post20170123-py2.7.egg/EGG-INFO/scripts/log2timeline.py", line 575, in ProcessSources
    worker_memory_limit=self._worker_memory_limit)
  File "/usr/local/lib/python2.7/dist-packages/plaso-1.5.2.post20170123-py2.7.egg/plaso/frontend/extraction_frontend.py", line 391, in ProcessSources
    worker_memory_limit=worker_memory_limit)
  File "/usr/local/lib/python2.7/dist-packages/plaso-1.5.2.post20170123-py2.7.egg/plaso/multi_processing/task_engine.py", line 770, in ProcessSources
    filter_find_specs=filter_find_specs)
  File "/usr/local/lib/python2.7/dist-packages/plaso-1.5.2.post20170123-py2.7.egg/plaso/multi_processing/task_engine.py", line 275, in _ProcessSources
    self._ScheduleTasks(storage_writer)
  File "/usr/local/lib/python2.7/dist-packages/plaso-1.5.2.post20170123-py2.7.egg/plaso/multi_processing/task_engine.py", line 407, in _ScheduleTasks
    self._MergeTaskStorage(storage_writer)
  File "/usr/local/lib/python2.7/dist-packages/plaso-1.5.2.post20170123-py2.7.egg/plaso/multi_processing/task_engine.py", line 205, in _MergeTaskStorage
    maximum_number_of_containers=self._MAXIMUM_NUMBER_OF_CONTAINERS)
  File "/usr/local/lib/python2.7/dist-packages/plaso-1.5.2.post20170123-py2.7.egg/plaso/storage/gzip_file.py", line 415, in MergeAttributeContainers
    self._AddAttributeContainer(attribute_container)
  File "/usr/local/lib/python2.7/dist-packages/plaso-1.5.2.post20170123-py2.7.egg/plaso/storage/gzip_file.py", line 348, in _AddAttributeContainer
    self._storage_writer.AddEvent(attribute_container)
  File "/usr/local/lib/python2.7/dist-packages/plaso-1.5.2.post20170123-py2.7.egg/plaso/storage/zip_file.py", line 3050, in AddEvent
    self._storage_file.AddEvent(event)
  File "/usr/local/lib/python2.7/dist-packages/plaso-1.5.2.post20170123-py2.7.egg/plaso/storage/zip_file.py", line 2346, in AddEvent
    self._WriteSerializedEvents()
  File "/usr/local/lib/python2.7/dist-packages/plaso-1.5.2.post20170123-py2.7.egg/plaso/storage/zip_file.py", line 2021, in _WriteSerializedEvents
    self._serialized_events_heap, self._event_stream_number)
  File "/usr/local/lib/python2.7/dist-packages/plaso-1.5.2.post20170123-py2.7.egg/plaso/storage/zip_file.py", line 2055, in _WriteSerializedEventsHeap
    entry_data_offset = data_stream.WriteEntry(entry_data)
  File "/usr/local/lib/python2.7/dist-packages/plaso-1.5.2.post20170123-py2.7.egg/plaso/storage/zip_file.py", line 565, in WriteEntry
    raise IOError(u'Unable to write data entry size value out of bounds.')
IOError: Unable to write data entry size value out of bounds.
2017-01-24 02:55:23,599 [WARNING] (Worker_00 ) PID:3141 <extractors> openxml unable to parse file: TSK:/Windows/System32/zipcontainer.dll with error: Invalid offset value less than zero.

Closing comment (2017-02-02T04:46:40Z): Re-run the test. Has the following error:
Traceback (most recent call last):
  File "/usr/local/bin/log2timeline.py", line 4, in <module>
    __import__('pkg_resources').run_script('plaso==1.5.2.post20170201', 'log2timeline.py')
  File "/usr/local/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 738, in run_script
    self.require(requires)[0].run_script(script_name, ns)
  File "/usr/local/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 1499, in run_script
    exec(code, namespace, namespace)
  File "/usr/local/lib/python2.7/dist-packages/plaso-1.5.2.post20170201-py2.7.egg/EGG-INFO/scripts/log2timeline.py", line 678, in <module>
    if not Main():
  File "/usr/local/lib/python2.7/dist-packages/plaso-1.5.2.post20170201-py2.7.egg/EGG-INFO/scripts/log2timeline.py", line 664, in Main
    tool.ProcessSources()
  File "/usr/local/lib/python2.7/dist-packages/plaso-1.5.2.post20170201-py2.7.egg/EGG-INFO/scripts/log2timeline.py", line 575, in ProcessSources
    worker_memory_limit=self._worker_memory_limit)
  File "/usr/local/lib/python2.7/dist-packages/plaso-1.5.2.post20170201-py2.7.egg/plaso/frontend/extraction_frontend.py", line 391, in ProcessSources
    worker_memory_limit=worker_memory_limit)
  File "/usr/local/lib/python2.7/dist-packages/plaso-1.5.2.post20170201-py2.7.egg/plaso/multi_processing/task_engine.py", line 769, in ProcessSources
    filter_find_specs=filter_find_specs)
  File "/usr/local/lib/python2.7/dist-packages/plaso-1.5.2.post20170201-py2.7.egg/plaso/multi_processing/task_engine.py", line 275, in _ProcessSources
    self._ScheduleTasks(storage_writer)
  File "/usr/local/lib/python2.7/dist-packages/plaso-1.5.2.post20170201-py2.7.egg/plaso/multi_processing/task_engine.py", line 404, in _ScheduleTasks
    if self._ScheduleTask(task):
  File "/usr/local/lib/python2.7/dist-packages/plaso-1.5.2.post20170201-py2.7.egg/plaso/multi_processing/task_engine.py", line 307, in _ScheduleTask
    self._task_queue.PushItem(task, block=False)
  File "/usr/local/lib/python2.7/dist-packages/plaso-1.5.2.post20170201-py2.7.egg/plaso/engine/zeromq_queue.py", line 775, in PushItem
    raise errors.QueueFull(exception)
plaso.lib.errors.QueueFull

Similar to #1036 ?
Rgds