internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: cowrie/cowrie

Issue:
TypeError in command "apt-get install"
**Describe the bug**

When using the command "apt-get install XY" while logged in per SSH in the cowrie honeypot, nothing happens.
If you look into the logs, the requested software couldn't be installed but a TypeError is thrown:

```Traceback (most recent call last):
	  File "/home/cowrie/github/cowrie/cowrie/shell/protocol.py", line 359, in call_command
	    HoneyPotBaseProtocol.call_command(self, pp, cmd, *args)
	  File "/home/cowrie/github/cowrie/cowrie/shell/protocol.py", line 216, in call_command
	    obj.start()
	  File "/home/cowrie/github/cowrie/cowrie/commands/apt.py", line 36, in start
	    self.do_install()
	  File "/home/cowrie/github/cowrie/cowrie-env/lib/python2.7/site-packages/twisted/internet/defer.py", line 1532, in unwindGenerator
	    return _inlineCallbacks(None, gen, Deferred())
	--- <exception caught here> ---
	  File "/home/cowrie/github/cowrie/cowrie-env/lib/python2.7/site-packages/twisted/internet/defer.py", line 1386, in _inlineCallbacks
	    result = g.send(result)
	  File "/home/cowrie/github/cowrie/cowrie/commands/apt.py", line 123, in do_install
	    'version': '{0}.{1}-{2}'.format(random.choice(0, 1), random.randint(1, 40), random.randint(1, 10)),
	exceptions.TypeError: choice() takes exactly 2 arguments (3 given)

 ```

I investigated the source and saw what is actually wrong:
```
'version': '{0}.{1}-{2}'.format(random.choice(0, 1), random.randint(1, 40), random.randint(1, 10)),
```
in which `random.choice(0, 1)` requires a sequence, so `random.choice([0, 1])` should work.

**To Reproduce**
Steps to reproduce the behavior:
1. Log in a cowrie honeypot per ssh: ssh -p 2222 root@honeypot
2. Try "sudo apt-get install vim"
3. See the error in the cowrie logs on your machine:  cowrie/logs/cowrie.log

**Expected behavior**

See some common apt-get messages, like "Reading package lists... Done\n", e.g. simulating to the attacker the command worked.

**Server (please complete the following information):**
 - OS: CentOS 7, x86_64
 - Python: Python 2.7.5 and Python 3.6.5

**Additional context**
-



Discussion:

Resolving patch:
diff --git a/cowrie/commands/apt.py b/cowrie/commands/apt.py
--- a/cowrie/commands/apt.py
+++ b/cowrie/commands/apt.py
@@ -120,7 +120,7 @@ def do_install(self, *args):
         packages = {}
         for y in [re.sub('[^A-Za-z0-9]', '', x) for x in self.args[1:]]:
             packages[y] = {
-                'version': '{0}.{1}-{2}'.format(random.choice(0, 1), random.randint(1, 40), random.randint(1, 10)),
+                'version': '{0}.{1}-{2}'.format(random.choice([0, 1]), random.randint(1, 40), random.randint(1, 10)),
                 'size': random.randint(100, 900)
             }
         totalsize = sum([packages[x]['size'] for x in packages])


Test patch:
Repository: cowrie/cowrie

Issue:
Mysql output has incorrect sql query in line 121
line 120-121 of src/cowrie/output/mysql.py show:
```
elif entry["eventid"] == 'cowrie.login.failed':
    self.simpleQuery('INSERT INTO `auth` (`session`, `success` `username`, `password`, `timestamp`) '
```
There is a missing comma between `success` & `username`. This results in failure to log failed login attempts to mysql.


Discussion:

Resolving patch:
diff --git a/src/cowrie/output/mysql.py b/src/cowrie/output/mysql.py
--- a/src/cowrie/output/mysql.py
+++ b/src/cowrie/output/mysql.py
@@ -118,7 +118,7 @@ def write(self, entry):
                              (entry["session"], 1, entry['username'], entry['password'], entry["time"]))
 
         elif entry["eventid"] == 'cowrie.login.failed':
-            self.simpleQuery('INSERT INTO `auth` (`session`, `success` `username`, `password`, `timestamp`) '
+            self.simpleQuery('INSERT INTO `auth` (`session`, `success`, `username`, `password`, `timestamp`) '
                              'VALUES (%s, %s, %s, %s, FROM_UNIXTIME(%s))',
                              (entry["session"], 0, entry['username'], entry['password'], entry["time"]))
 


Test patch:
Repository: cowrie/cowrie

Issue:
adding root:x:!password to userdb.txt doesn't exclude root/password as valid credentials
Fresh install.

I tried to exclude 'password' or 'abc123' from valid passwords for the user root

Now file looks like 
```
root:x:!root
root:x:!123456
root:x:!password
root:x:*
```

Retarted cowrie, but no way to deny login with root/password credentials

Maybe, some sort of problem with new regexp checking?


Discussion:
That used to work, maybe that stopped working with the new regex patch indeed.
Hello @supriyo-biswas 
In your patch https://github.com/micheloosterhof/cowrie/pull/763 we lost the ability to negate passwords starting with '!'. Any ideas on how to do this with regex?
Yep, experiencing the same issue here. I looked through the file and nothing stood out to me. If this doesn't get resolved I'll have more time to look at it tomorrow night.
seems to be an ordering issue, #802 should fix it
Same problem for me but not solve by the patch
Same problem, not solved by patch
Reopened.
@micheloosterhof I think I found what is causing the problem here.

In [auth.py Line 101](https://github.com/cowrie/cowrie/blob/master/src/cowrie/core/auth.py#L101) we are checking
```python
if passwd[0] == b'!':
```
but the problem is that `passwd` is a byte so `passwd[0]` would give us the value of the character on the 0 indexes.

Ex:

```python

>>> p
b'!123456'

>>> p[0]
33

>>> ord("!")
33

```

So instead of checking for `b"!"` we need to check for it's `ord("!")`.
```python
if passwd[0] == ord("!"):
```

P.S: This is again python2/python3 issue :)

Resolving patch:
diff --git a/src/cowrie/core/auth.py b/src/cowrie/core/auth.py
--- a/src/cowrie/core/auth.py
+++ b/src/cowrie/core/auth.py
@@ -98,7 +98,7 @@ def adduser(self, login, passwd):
         """
         login = self.re_or_str(login)
 
-        if passwd[0] == b'!':
+        if passwd[0] == ord("!"):
             policy = False
             passwd = passwd[1:]
         else:


Test patch:
Repository: cowrie/cowrie

Issue:
Reverse DNS
##### ISSUE TYPE
 - Feature Idea

##### DESCRIPTION
Add the ability to enable Reverse DNS lookups. At the moment a detection contains the IP of the attacker. We would also like to see the Reverse DNS of the attacker.

We can lookup the reverse DNS afterwards in Elasticsearch or in the logstash pipeline, but we need to use the local DNS servers of the device which runs Cowrie in order to see the internal reverse DNS/hostnames.


Discussion:
@micheloosterhof @ZeNiRe Can you please give me some more information on this?
I mean we can add an option in the config file to enable and disable this and for actually doing this with python we can either do something like

```python
import socket

reverse = socket.gethostbyaddr(IP)[0]
```
or we can use something more powerful like [dnspython](https://github.com/rthalley/dnspython) to get more info.

The thing I am confused about is where should this be added in cowrie as in which files would have to be changed to actually implement this.

Also if you think there's some other/better way to do it then please let me know.

Resolving patch:
diff --git a/src/cowrie/output/reversedns.py b/src/cowrie/output/reversedns.py
new file mode 100644
--- /dev/null
+++ b/src/cowrie/output/reversedns.py
@@ -0,0 +1,68 @@
+from __future__ import absolute_import, division
+
+from twisted.names import client
+from twisted.python import log
+
+import cowrie.core.output
+from cowrie.core.config import CONFIG
+
+
+class Output(cowrie.core.output.Output):
+    """
+    Output plugin used for reverse DNS lookup
+    """
+
+    def __init__(self):
+        self.timeout = CONFIG.getint('output_reversedns', 'timeout', fallback=3)
+        cowrie.core.output.Output.__init__(self)
+
+    def start(self):
+        """
+        Start Output Plugin
+        """
+        pass
+
+    def stop(self):
+        """
+        Stop Output Plugin
+        """
+        pass
+
+    def write(self, entry):
+        if entry['eventid'] == 'cowrie.session.connect':
+            self.reversedns(entry['src_ip'])
+
+    def reversedns(self, addr):
+        """
+        Perform a reverse DNS lookup on an IP
+
+        Arguments:
+            addr -- IPv4 Address
+        """
+        ptr = self.reverseNameFromIPAddress(addr)
+        d = client.lookupPointer(ptr, timeout=self.timeout)
+
+        def cbError(failure):
+            log.msg("reversedns: Error in lookup")
+            failure.printTraceback()
+
+        def processResult(result):
+            """
+            Process the lookup result
+            """
+            RR = result[0][0]
+            log.msg("Reverse DNS record for ip={0}: {1}".format(
+                addr, RR.payload))
+
+        d.addCallback(processResult)
+        d.addErrback(cbError)
+        return d
+
+    def reverseNameFromIPAddress(self, address):
+        """
+        Reverse the IPv4 address and append in-addr.arpa
+
+        Arguments:
+            address {str} -- IP address that is to be reversed
+        """
+        return '.'.join(reversed(address.split('.'))) + '.in-addr.arpa'


Test patch:
Repository: cowrie/cowrie

Issue:
Can't enter using "oracle" or "tomcat" users
Hello.
I'm using stock userdb.txt file, where "oracle" and "tomcat" users are defined with any password. When using these users, the ssh client gives me an error of "Permission denied (publickey,password)" after entering three times any password.

The ugly thing is that in cowrie.json file appear entries of "cowrie.login.success" type with the message "login attempt [oracle/password] suceeded", which is, obviously, incorrect.

Thanks a lot


Discussion:

Resolving patch:
diff --git a/src/cowrie/shell/pwd.py b/src/cowrie/shell/pwd.py
--- a/src/cowrie/shell/pwd.py
+++ b/src/cowrie/shell/pwd.py
@@ -117,7 +117,7 @@ def setpwentry(self, name):
         """
 
         # ensure consistent uid and gid
-        seed_id = crc32(name)
+        seed_id = crc32(name.encode("utf-8"))
         seed(seed_id)
 
         e = {}


Test patch:
Repository: cowrie/cowrie

Issue:
greynoise should catch timeout error
```2019-04-08T03:12:05.460833Z [twisted.internet.defer#critical] Unhandled error in Deferred:
2019-04-08T03:12:05.462257Z [twisted.internet.defer#critical]
        Traceback (most recent call last):
        --- <exception caught here> ---
          File "/home/cowrie/cowrie/src/cowrie/output/greynoise.py", line 65, in scanip
            headers=headers)
        twisted.internet.error.TimeoutError: User timeout caused connection failure.
```



Discussion:
I think this can be fixed by passing the `timeout=10` to the treq request we are sending in [greynoise.py Line#62](https://github.com/cowrie/cowrie/blob/master/src/cowrie/output/greynoise.py#L62).

So the requests will become:
```python
        response = yield treq.post(
            url=gnUrl,
            data=fields,
            headers=headers,
     			 timeout=10)
```

Resolving patch:
diff --git a/src/cowrie/output/greynoise.py b/src/cowrie/output/greynoise.py
--- a/src/cowrie/output/greynoise.py
+++ b/src/cowrie/output/greynoise.py
@@ -6,7 +6,7 @@
 
 import treq
 
-from twisted.internet import defer
+from twisted.internet import defer, error
 from twisted.python import log
 
 import cowrie.core.output
@@ -59,10 +59,15 @@ def message(query):
         headers = ({'User-Agent': [COWRIE_USER_AGENT]})
         fields = {'key': self.apiKey, 'ip': entry['src_ip']}
 
-        response = yield treq.post(
-            url=gnUrl,
-            data=fields,
-            headers=headers)
+        try:
+            response = yield treq.post(
+                url=gnUrl,
+                data=fields,
+                headers=headers,
+                timeout=10)
+        except (defer.CancelledError, error.ConnectingCancelledError, error.DNSLookupError):
+            log.msg("GreyNoise requests timeout")
+            return
 
         if response.code != 200:
             rsp = yield response.text()
@@ -72,13 +77,14 @@ def message(query):
         j = yield response.json()
         if self.debug:
             log.msg("greynoise: debug: "+repr(j))
-            if j['status'] == "ok":
-                if "all" not in self.tags:
-                    for query in j['records']:
-                        if query['name'] in self.tags:
-                            message(query)
-                else:
-                    for query in j['records']:
+
+        if j['status'] == "ok":
+            if "all" not in self.tags:
+                for query in j['records']:
+                    if query['name'] in self.tags:
                         message(query)
             else:
-                log.msg("greynoise: no results for for IP {0}".format(entry['src_ip']))
+                for query in j['records']:
+                    message(query)
+        else:
+            log.msg("greynoise: no results for for IP {0}".format(entry['src_ip']))


Test patch:
Repository: cowrie/cowrie

Issue:
`ls -l` user/group names need justification
`ls -l` does not properly pad the user/group names

**To Reproduce**
Steps to reproduce the behaviour:
1. ssh into a cowrie instance
2. `ls -l` on a directory with more than one user/group
3. the user names and group names don't line up between files

**Expected behaviour**
Nice justified columns of user/group names



Discussion:

Resolving patch:
diff --git a/src/cowrie/commands/ls.py b/src/cowrie/commands/ls.py
--- a/src/cowrie/commands/ls.py
+++ b/src/cowrie/commands/ls.py
@@ -112,9 +112,17 @@ def do_ls_normal(self, path):
     def do_ls_l(self, path):
         files = self.get_dir_files(path)
 
-        largest = 0
+        filesize_str_extent = 0
         if len(files):
-            largest = max([x[fs.A_SIZE] for x in files])
+            filesize_str_extent = max([len(str(x[fs.A_SIZE])) for x in files])
+
+        user_name_str_extent = 0
+        if len(files):
+            user_name_str_extent = max([len(self.uid2name(x[fs.A_UID])) for x in files])
+
+        group_name_str_extent = 0
+        if len(files):
+            group_name_str_extent = max([len(self.gid2name(x[fs.A_GID])) for x in files])
 
         for file in files:
             if file[fs.A_NAME].startswith('.') and not self.showHidden:
@@ -167,9 +175,9 @@ def do_ls_l(self, path):
 
             line = '%s 1 %s %s %s %s %s%s' % \
                 (perms,
-                 self.uid2name(file[fs.A_UID]),
-                 self.gid2name(file[fs.A_GID]),
-                 str(file[fs.A_SIZE]).rjust(len(str(largest))),
+                 self.uid2name(file[fs.A_UID]).ljust(user_name_str_extent),
+                 self.gid2name(file[fs.A_GID]).ljust(group_name_str_extent),
+                 str(file[fs.A_SIZE]).rjust(filesize_str_extent),
                  time.strftime('%Y-%m-%d %H:%M', ctime),
                  file[fs.A_NAME],
                  linktarget)


Test patch:
Repository: cowrie/cowrie

Issue:
Erroneous handling of MySQL errors
The MySQL plug-in (`src/cowrie/output/mysql.py`) has code like this:

```python
        except MySQLdb.OperationalError as e:
            if e[0] not in (2003, 2006, 2013):
                raise e
```

From there, this code has been inherited by another honeypot of mine. Yesterday I accidentally hit it by running the honeypot in a Docker container that didn't have access to the MySQL server and forgetting to disable the MySQL plug-in in the config file, so the exception occurred.

Well, apparently, you can't access the error code like this (`e[0]`), or you get an error that you're trying to index something that is not index-able. The proper way to access the error code is by using `e.args[0]`. See, for instance, [this example](https://github.com/B-ROY/TESTGIT/blob/40221cf254c90d37d21afb981635740aebf11949/base/site-packages/django/db/backends/mysql/base.py#L126).

Man, this bug must have been there since forever... At least since 2008, judging by the comments in that code, although the links there have died by bit rot and are no longer accessible. The error is [present in Kippo](https://github.com/desaster/kippo/blob/master/kippo/dblog/mysql.py), too. Maybe originally this is how MySQLdb returned the error code?


Discussion:
Could this be a Python2/3 thing?


Resolving patch:
diff --git a/src/cowrie/output/mysql.py b/src/cowrie/output/mysql.py
--- a/src/cowrie/output/mysql.py
+++ b/src/cowrie/output/mysql.py
@@ -32,7 +32,7 @@ def _runInteraction(self, interaction, *args, **kw):
             return adbapi.ConnectionPool._runInteraction(
                 self, interaction, *args, **kw)
         except MySQLdb.OperationalError as e:
-            if e[0] not in (2003, 2006, 2013):
+            if e.args[0] not in (2003, 2006, 2013):
                 raise e
             log.msg("RCP: got error {0}, retrying operation".format(e))
             conn = self.connections.get(self.threadID())


Test patch:
Repository: cowrie/cowrie

Issue:
Programming errors in the InfluxDB plugin
Something definitely seems wrong with the InfluxDB output plugin... I can see at least two possible problems:

1. The error handling

```python
        except InfluxDBClientError as e:
            log.msg("output_influx: I/O error({0}): '{1}'".format(
                e.errno, e.strerror))
            return
```

seems wrong to me. If you inspect the source of the Python [InfluxDB client](https://github.com/influxdata/influxdb-python/blob/master/influxdb/exceptions.py), the `InfluxDBClientError` class does not have `errno` and `strerror` fields. I think the plugin should use `e.code` and `e.message` instead respectively.

2. In the plugin, there is some code like this:

```python
        m = {
            'measurement': eventid.replace('.', '_'),
            'tags': {
                'session': entry['session'],
                'src_ip': entry['src_ip']
            },
            'fields': {
                'sensor': self.sensor
            },
        }
```

However `self.sensor` is not defined anywhere. I think it should be `entry['sensor']` instead.


Discussion:
So `self.sensor` is set in `__init__()` in `src/cowrie/core/output.py` so that should be inherited.
For the other fields I think are you are correct. 

Resolving patch:
diff --git a/src/cowrie/output/influx.py b/src/cowrie/output/influx.py
--- a/src/cowrie/output/influx.py
+++ b/src/cowrie/output/influx.py
@@ -23,7 +23,7 @@ def start(self):
             self.client = InfluxDBClient(host=host, port=port, ssl=ssl, verify_ssl=ssl)
         except InfluxDBClientError as e:
             log.msg("output_influx: I/O error({0}): '{1}'".format(
-                e.errno, e.strerror))
+                e.code, e.message))
             return
 
         if self.client is None:


Test patch:
Repository: cowrie/cowrie

Issue:
MySQL error handling
I have created several honeypots for other protocols (Android Debug Bridge, Citrix Netscaler, Elasticsearch, Internet Printing Protocol). In all of them I use Cowrie's idea of output plugins. However, the attack traffic for these protocols is much much lower than for Telnet/SSH. Probably for this reason, the MySQL plugin often fails with the dreaded "(2006, 'MySQL server has gone away')" error. (Yes, sometimes more than 8 hours can pass between attacks.)

What bothered me is that the plugin failed with a stack trace. This isn't supposed to happen, because it intercepts this error just like Cowrie's MySQL plugin and tries to re-establish the connection. Then I noticed that the error that is being reported is not `MySQLdb.OperationalError`, as the code intercepts. Instead, it is `MySQLdb._exceptions.OperationalError`.

And, indeed, linting the code complains that the `MySLdb` module does not have a member named `OperationalError`. Similarly, there is no `MySQLdb.Error` - but there seems to be `MySQLdb._exceptons.Error` instead.

Shouldn't these be changed in Cowrie's MySQL module?

Also, from the [documentation of MySQLdb](https://mysqlclient.readthedocs.io/FAQ.html#my-data-disappeared-or-won-t-go-away):

> Starting with 1.2.0, MySQLdb disables autocommit by default, as required by the DB-API standard (PEP-249). If you are using InnoDB tables or some other type of transactional table type, you’ll need to do connection.commit() before closing the connection, or else none of your changes will be written to the database.

Shouldn't the `stop()` method of Cowrie's MySQL plugin issue a `self.db.commit()` before closing the connection? Or maybe set `self.db.autocommit(True)` when opening it?


Discussion:
Thanks for reporting! Those may be leftovers from the old `MySQL-python`, since we now moved to `mysqlclient` for Py3+

Resolving patch:
diff --git a/src/cowrie/output/mysql.py b/src/cowrie/output/mysql.py
--- a/src/cowrie/output/mysql.py
+++ b/src/cowrie/output/mysql.py
@@ -31,7 +31,7 @@ def _runInteraction(self, interaction, *args, **kw):
         try:
             return adbapi.ConnectionPool._runInteraction(
                 self, interaction, *args, **kw)
-        except MySQLdb.OperationalError as e:
+        except (MySQLdb.OperationalError, MySQLdb._exceptions.OperationalError) as e:
             if e.args[0] not in (2003, 2006, 2013):
                 raise e
             log.msg("RCP: got error {0}, retrying operation".format(e))
@@ -65,10 +65,11 @@ def start(self):
                 cp_reconnect=True,
                 use_unicode=True
             )
-        except MySQLdb.Error as e:
+        except (MySQLdb.Error, MySQLdb._exceptons.Error) as e:
             log.msg("output_mysql: Error %d: %s" % (e.args[0], e.args[1]))
 
     def stop(self):
+        self.db.commit()
         self.db.close()
 
     def sqlerror(self, error):


Test patch:
Repository: cowrie/cowrie

Issue:
bash: rm not found
**Describe the bug**
In my ttylogs, I see the following error code:

-bash: rm: command not found

**To Reproduce**
Steps to reproduce the behavior:
1. Log in
2. Create a file
3. Try to delete a file (rm -rf < filename >)

**Expected behavior**
An error saying that the file could not be deleted?

**Server (please complete the following information):**
 - OS: Linux nanopineo2 5.8.6-sunxi64 #20.08.2 SMP Fri Sep 4 08:52:31 CEST 2020 aarch64 GNU/Linux
 - Python: Python 3.7.3 (venv)

**Additional context**
Add any other context about the problem here.



Discussion:

Resolving patch:
diff --git a/src/cowrie/commands/fs.py b/src/cowrie/commands/fs.py
--- a/src/cowrie/commands/fs.py
+++ b/src/cowrie/commands/fs.py
@@ -89,6 +89,7 @@ def handle_CTRL_D(self):
 
 
 commands['/bin/grep'] = command_grep
+commands['grep'] = command_grep
 commands['/bin/egrep'] = command_grep
 commands['/bin/fgrep'] = command_grep
 
@@ -156,6 +157,7 @@ def handle_CTRL_D(self):
 
 commands['/bin/tail'] = command_tail
 commands['/usr/bin/tail'] = command_tail
+commands['tail'] = command_tail
 
 
 class command_head(HoneyPotCommand):
@@ -215,6 +217,7 @@ def handle_CTRL_D(self):
 
 commands['/bin/head'] = command_head
 commands['/usr/bin/head'] = command_head
+commands['head'] = command_head
 
 
 class command_cd(HoneyPotCommand):
@@ -350,6 +353,7 @@ def call(self):
 
 
 commands['/bin/rm'] = command_rm
+commands['rm'] = command_rm
 
 
 class command_cp(HoneyPotCommand):
@@ -422,6 +426,7 @@ def resolv(pname):
 
 
 commands['/bin/cp'] = command_cp
+commands['cp'] = command_cp
 
 
 class command_mv(HoneyPotCommand):
@@ -489,6 +494,7 @@ def resolv(pname):
 
 
 commands['/bin/mv'] = command_mv
+commands['mv'] = command_mv
 
 
 class command_mkdir(HoneyPotCommand):
@@ -544,6 +550,7 @@ def call(self):
 
 
 commands['/bin/rmdir'] = command_rmdir
+commands['rmdir'] = command_rmdir
 
 
 class command_pwd(HoneyPotCommand):
@@ -556,6 +563,7 @@ def call(self):
 
 
 commands['/bin/pwd'] = command_pwd
+commands['pwd'] = command_pwd
 
 
 class command_touch(HoneyPotCommand):
diff --git a/src/cowrie/commands/wc.py b/src/cowrie/commands/wc.py
--- a/src/cowrie/commands/wc.py
+++ b/src/cowrie/commands/wc.py
@@ -117,3 +117,4 @@ def handle_CTRL_D(self):
 
 commands['/usr/bin/wc'] = command_wc
 commands['/bin/wc'] = command_wc
+commands['wc'] = command_wc


Test patch: