Repository: cowrie/cowrie

Issue:
Mysql error? [twisted.internet.defer#critical] Unhandled error in Deferred
```
2021-01-11T16:54:56.352309Z [CowrieTelnetTransport,2,41.13.224.97] login attempt [b'root'/b'5up'] succeeded
2021-01-11T16:54:56.353787Z [CowrieTelnetTransport,2,41.13.224.97] Initialized emulated server as architecture: linux-x64-lsb
2021-01-11T16:54:56.354732Z [twisted.internet.defer#critical] Unhandled error in Deferred:
2021-01-11T16:54:56.354941Z [twisted.internet.defer#critical]
        Traceback (most recent call last):
          File "/home/cowrie/cowrie/cowrie-env/lib/python3.8/site-packages/twisted/logger/_legacy.py", line 93, in __call__
            self.legacyObserver(event)
          File "/home/cowrie/cowrie/src/cowrie/core/output.py", line 218, in emit
            self.write(ev)
          File "/home/cowrie/cowrie/cowrie-env/lib/python3.8/site-packages/twisted/internet/defer.py", line 1613, in unwindGenerator
            return _cancellableInlineCallbacks(gen)
          File "/home/cowrie/cowrie/cowrie-env/lib/python3.8/site-packages/twisted/internet/defer.py", line 1529, in _cancellableInlineCallbacks
            _inlineCallbacks(None, g, status)
        --- <exception caught here> ---
          File "/home/cowrie/cowrie/cowrie-env/lib/python3.8/site-packages/twisted/internet/defer.py", line 1418, in _inlineCallbacks
            result = g.send(result)
          File "/home/cowrie/cowrie/src/cowrie/output/mysql.py", line 147, in write
            (entry["session"], entry["time"], entry['url'], entry['outfile'], entry['shasum']))
        builtins.KeyError: 'url'

2021-01-11T16:54:56.413032Z [CowrieTelnetTransport,2,41.13.224.97] Warning: state changed and new state returned
```


Discussion:
I saw this issue too. I suppose, this happens when a visitor uploads a file to your honeypot using scp. In this case, there is clearly no URL defined. And the error here is raised exactly because there is no `url` key in the `entry` object.

I fixed it by replacing `entry["url"]` with `entry.get("url", "")`. Now if there is no URL present in the event, it will be simply set to an empty value.

A code snippet from my `src/cowrie/output/mysql.py`:
```
elif entry["eventid"] == 'cowrie.session.file_download':
    self.simpleQuery('INSERT INTO `downloads` (`session`, `timestamp`, `url`, `outfile`, `shasum`) '
                        'VALUES (%s, FROM_UNIXTIME(%s), %s, %s, %s)',
                        (entry["session"], entry["time"], entry.get("url", ""), entry['outfile'], entry['shasum']))

elif entry["eventid"] == 'cowrie.session.file_download.failed':
    self.simpleQuery('INSERT INTO `downloads` (`session`, `timestamp`, `url`, `outfile`, `shasum`) '
                        'VALUES (%s, FROM_UNIXTIME(%s), %s, %s, %s)',
                        (entry["session"], entry["time"], entry.get("url", ""), 'NULL', 'NULL'))
```

I can make a pull request if there is no better idea how to solve this.

Resolving patch:
diff --git a/src/cowrie/output/mysql.py b/src/cowrie/output/mysql.py
--- a/src/cowrie/output/mysql.py
+++ b/src/cowrie/output/mysql.py
@@ -144,12 +144,12 @@ def write(self, entry):
         elif entry["eventid"] == 'cowrie.session.file_download':
             self.simpleQuery('INSERT INTO `downloads` (`session`, `timestamp`, `url`, `outfile`, `shasum`) '
                              'VALUES (%s, FROM_UNIXTIME(%s), %s, %s, %s)',
-                             (entry["session"], entry["time"], entry['url'], entry['outfile'], entry['shasum']))
+                             (entry["session"], entry["time"], entry.get("url", ""), entry['outfile'], entry['shasum']))
 
         elif entry["eventid"] == 'cowrie.session.file_download.failed':
             self.simpleQuery('INSERT INTO `downloads` (`session`, `timestamp`, `url`, `outfile`, `shasum`) '
                              'VALUES (%s, FROM_UNIXTIME(%s), %s, %s, %s)',
-                             (entry["session"], entry["time"], entry['url'], 'NULL', 'NULL'))
+                             (entry["session"], entry["time"], entry.get("url", ""), 'NULL', 'NULL'))
 
         elif entry["eventid"] == 'cowrie.session.file_upload':
             self.simpleQuery('INSERT INTO `downloads` (`session`, `timestamp`, `url`, `outfile`, `shasum`) '


Test patch: