Repository: google/timesketch

Issue:
unknown logged error 
**Describe the bug**
celery worker log file is filled with 6913766 line of the following event's:
```
[2021-03-08 00:28:33,251] timesketch.elasticsearch/ERROR Errors while attempting to upload events.
[2021-03-08 00:28:33,252] timesketch.elasticsearch/ERROR Unable to upload document: (unable to get doc id) to index N/A - [0] <Unknown Type> No reason given [Unknown Type/Unkown Detailed Reason].
``` 

**To Reproduce**
Steps to reproduce the behavior:
1. upload a large plaso file.
2. run analyzers.


**Expected behavior**
A clear description of what caused the error to be logged.

**Screenshots**
If applicable, add screenshots to help explain your problem.

**Desktop (please complete the following information):**

   -OS:Ubuntu 20.04.2 LTS
   -Browser : Firefox
   -Version: 86.0


**Additional context**
-Timesketch docker version: 20210224
-worker.log:
```
[2021-03-08 00:28:31,208] timesketch.tasks/INFO [ssh_bruteforce_sessionizer] result: Sessionizing completed, number of session created: 0
[2021-03-08 00:28:31,212] celery.worker.strategy/INFO Received task: timesketch.lib.tasks.run_sketch_analyzer[c7b2dd57-5ae3-46dd-a020-c90a7b2c7388]  
[2021-03-08 00:28:31,213] celery.app.trace/INFO Task timesketch.lib.tasks.run_sketch_analyzer[80edd1a4-13db-47b2-9f80-b65e3e071077] succeeded in 0.052561102998879505s: '3d179a39c9864dd9a2ab37b9cb3e66bb'
[2021-03-08 00:28:33,251] timesketch.elasticsearch/ERROR Errors while attempting to upload events.
[2021-03-08 00:28:33,252] timesketch.elasticsearch/ERROR Unable to upload document: (unable to get doc id) to index N/A - [0] <Unknown Type> No reason given [Unknown Type/Unkown Detailed Reason]
.
.
.
[2021-03-08 02:24:51,267] timesketch.elasticsearch/ERROR Unable to upload document: (unable to get doc id) to index N/A - [0] <Unknown Type> No reason given [Unknown Type/Unkown Detailed Reason]
[2021-03-08 02:24:51,230] timesketch.elasticsearch/ERROR Errors while attempting to upload events.
```



Discussion:
ok, so few questions

a) how did you import the large plaso file? Importer, web ui?

b) can you include the logs from the actual ingestion of the file?

c) is the timeline properly ingested into TS, as in can you view the events in the ui at all?

d) what docker config are you using, the release with the latest tag or the dev docker config?
please find below answers:
a) how did you import the large plaso file? Importer, web ui?
using the API.
b) can you include the logs from the actual ingestion of the file?
you can find below the ingestion logs it was okay but as you can see there is Elasticsearch read timeout error here and there in the log file can i change the read timeout value?
```
[2021-03-07 21:35:41,789] celery.worker.strategy/INFO Received task: timesketch.lib.tasks.run_plaso[99dfea64-4e3d-4877-bce2-bca17fa81f8b]  
[2021-03-07 21:35:42,563] timesketch.tasks/INFO Index timeline [***********] to index [3d179a39c9864dd9a2ab37b9cb3e66bb] (source: plaso)
[2021-03-07 21:58:09,626] celery.worker.strategy/INFO Received task: timesketch.lib.tasks.run_plaso[ca8aab56-0710-4707-8ee3-452fd08104a3]  
[2021-03-07 21:58:09,962] timesketch.tasks/INFO Index timeline [***********] to index [eae371753701475dbf517fcf361385dd] (source: plaso)
[2021-03-07 22:41:19,958] celery.app.trace/INFO Task timesketch.lib.tasks.run_plaso[99dfea64-4e3d-4877-bce2-bca17fa81f8b] succeeded in 3938.1637521139996s: '3d179a39c9864dd9a2ab37b9cb3e66bb'
[2021-03-07 23:02:45,897] celery.app.trace/INFO Task timesketch.lib.tasks.run_plaso[ca8aab56-0710-4707-8ee3-452fd08104a3] succeeded in 3876.267387619s: 'eae371753701475dbf517fcf361385dd'
[2021-03-07 23:05:14,784] celery.worker.strategy/INFO Received task: timesketch.lib.tasks.run_sketch_init[04d6b021-bc24-412e-be9b-a77a3f2d7406]  
[2021-03-07 23:05:14,821] celery.app.trace/INFO Task timesketch.lib.tasks.run_sketch_init[04d6b021-bc24-412e-be9b-a77a3f2d7406] succeeded in 0.034461408999050036s: '3d179a39c9864dd9a2ab37b9cb3e66bb'
[2021-03-07 23:05:14,822] celery.worker.strategy/INFO Received task: timesketch.lib.tasks.run_sketch_analyzer[a0782da0-977c-46e4-9a12-9c497bf10b6f]  
[2021-03-07 23:05:15,186] celery.worker.strategy/INFO Received task: timesketch.lib.tasks.run_sketch_init[428fe155-c221-407a-a942-a56883a99277]  
[2021-03-07 23:05:15,227] celery.app.trace/INFO Task timesketch.lib.tasks.run_sketch_init[428fe155-c221-407a-a942-a56883a99277] succeeded in 0.03964604499924462s: 'eae371753701475dbf517fcf361385dd'
[2021-03-07 23:05:15,228] celery.worker.strategy/INFO Received task: timesketch.lib.tasks.run_sketch_analyzer[010fb412-d523-40d9-841a-d7a856ea1607]  
[2021-03-07 23:05:35,246] timesketch.analyzers.sigma_tagger/ERROR Timeout executing search for appframework_django_exceptions.yml: ConnectionTimeout caused by - ReadTimeoutError(HTTPConnectionPool(host='elasticsearch01', port=9200): Read timed out. (read timeout=10)) waiting for 10 seconds
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 421, in _make_request
    six.raise_from(e, None)
  File "<string>", line 3, in raise_from
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 416, in _make_request
    httplib_response = conn.getresponse()
  File "/usr/lib/python3.8/http/client.py", line 1347, in getresponse
    response.begin()
  File "/usr/lib/python3.8/http/client.py", line 307, in begin
    version, status, reason = self._read_status()
  File "/usr/lib/python3.8/http/client.py", line 268, in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
  File "/usr/lib/python3.8/socket.py", line 669, in readinto
    return self._sock.recv_into(b)
socket.timeout: timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/elasticsearch/connection/http_urllib3.py", line 241, in perform_request
    response = self.pool.urlopen(
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 719, in urlopen
    retries = retries.increment(
  File "/usr/lib/python3/dist-packages/urllib3/util/retry.py", line 376, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/usr/local/lib/python3.8/dist-packages/six.py", line 693, in reraise
    raise value
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 665, in urlopen
    httplib_response = self._make_request(
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 423, in _make_request
    self._raise_timeout(err=e, url=url, timeout_value=read_timeout)
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 330, in _raise_timeout
    raise ReadTimeoutError(
urllib3.exceptions.ReadTimeoutError: HTTPConnectionPool(host='elasticsearch01', port=9200): Read timed out. (read timeout=10)
```  
c) is the timeline properly ingested into TS, as in can you view the events in the ui at all?
Yes the timelines are properly ingested into TS i can search and work with the data.
d) what docker config are you using, the release with the latest tag or the dev docker config?
the current docker config is the release with the latest tag as seen below:
```
$ tsctl version
Timesketch version: 20210224
```
ok, so you are getting timeout errors, let's look at the time out value here, see if we can at least increase that, thought we did that already, probably not everywhere.
Maybe this is were the timeout value could be increased?
https://github.com/google/timesketch/blob/0def3cb2d41ed032c21d4909f5c8262e30b9f33d/timesketch/lib/datastores/elastic.py#L434-L436

Hm how many Sigma rules do you have in your system?
 This timeout can happen if you have to many, as every rule is creating a new query.
i had in the system all sigma rules covered in the following issue #1532 

Resolving patch:
diff --git a/timesketch/lib/datastores/elastic.py b/timesketch/lib/datastores/elastic.py
--- a/timesketch/lib/datastores/elastic.py
+++ b/timesketch/lib/datastores/elastic.py
@@ -116,6 +116,7 @@ def __init__(self, host='127.0.0.1', port=9200):
         self.password = current_app.config.get('ELASTIC_PASSWORD', 'pass')
         self.ssl = current_app.config.get('ELASTIC_SSL', False)
         self.verify = current_app.config.get('ELASTIC_VERIFY_CERTS', True)
+        self.timeout = current_app.config.get('ELASTIC_TIMEOUT', 10)
 
         parameters = {}
         if self.ssl:
@@ -124,6 +125,8 @@ def __init__(self, host='127.0.0.1', port=9200):
 
         if self.user and self.password:
             parameters['http_auth'] = (self.user, self.password)
+        if self.timeout:
+            parameters['timeout'] = self.timeout
 
         self.client = Elasticsearch(
             [{'host': host, 'port': port}], **parameters)


Test patch: