internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: ansible/ansible

Issue:
hash filter is broken with non-ascii and on python3 due to lack of encoding
##### ISSUE TYPE
 - Bug Report

##### COMPONENT NAME
Hashing filters

##### ANSIBLE VERSION
```
ansible 2.2.1.0
  config file = 
  configured module search path = Default w/o overrides
```

##### CONFIGURATION
NA

##### OS / ENVIRONMENT
CentOS 7, Python 3.5

##### SUMMARY
An ASCII file that was slurped could not be hashed. It fails with `Unicode-objects must be encoded before hashing`

##### STEPS TO REPRODUCE

- touch a file `openssl.cnf` with some plaintext data
- run the below playbook (client should run with Python 3.5)

```yaml
---

- hosts: localhost
  tasks:
    - name: slurp openssl.cnf content
      slurp:
        src: "openssl.cnf"
      register: openssl_cnf
      run_once: true

    - name: record openssl.cnf hash
      set_fact:
        openssl_cnf_sha1: "{{ openssl_cnf.content | hash('sha1') }}"
```

##### EXPECTED RESULTS
It's expected the openssl_cnf_sha1 fact is set, which is the case when run with python 2.
```
TASK [record openssl.cnf hash] *************************************************
ok: [localhost] => {"ansible_facts": {"openssl_cnf_sha1": "8a7426daafe2edf55e2adeb3500d2628915a171d"}, "changed": false}
```

##### ACTUAL RESULTS
The hashing filter failed with the below error:

```
TASK [record openssl.cnf hash] *************************************************
fatal: [localhost]: FAILED! => {"failed": true, "msg": "Unexpected templating type error occurred on ({{ openssl_cnf.content | hash('sha1') }}): Unicode-objects must be encoded before hashing"}

```


Discussion:

Resolving patch:
diff --git a/lib/ansible/plugins/filter/core.py b/lib/ansible/plugins/filter/core.py
--- a/lib/ansible/plugins/filter/core.py
+++ b/lib/ansible/plugins/filter/core.py
@@ -48,7 +48,7 @@
 from ansible.compat.six import iteritems, string_types, integer_types
 from ansible.compat.six.moves import reduce
 from ansible.compat.six.moves import shlex_quote
-from ansible.module_utils._text import to_text
+from ansible.module_utils._text import to_bytes, to_text
 from ansible.parsing.yaml.dumper import AnsibleDumper
 from ansible.utils.hashing import md5s, checksum_s
 from ansible.utils.unicode import unicode_wrap
@@ -236,7 +236,7 @@ def get_hash(data, hashtype='sha1'):
     except:
         return None
 
-    h.update(data)
+    h.update(to_bytes(data, errors='surrogate_then_strict'))
     return h.hexdigest()
 
 def get_encrypted_password(password, hashtype='sha512', salt=None):


Test patch:
diff --git a/test/integration/targets/filters/tasks/main.yml b/test/integration/targets/filters/tasks/main.yml
--- a/test/integration/targets/filters/tasks/main.yml
+++ b/test/integration/targets/filters/tasks/main.yml
@@ -119,3 +119,9 @@
   assert:
     that:
       - "users | json_query('[*].hosts[].host') == ['host_a', 'host_b', 'host_c', 'host_d']"
+
+- name: Test hash filter
+  assert:
+    that:
+      - '"{{ "hash" | hash("sha1") }}" == "2346ad27d7568ba9896f1b7da6b5991251debdf2"'
+      - '"{{ "café" | hash("sha1") }}" == "f424452a9673918c6f09b0cdd35b20be8e6ae7d7"'
Repository: ansible/ansible

Issue:
Persistent Connections: raw with unicode arg and output
##### ISSUE TYPE
 - Bug Report

##### COMPONENT NAME
persistent connection manager

##### ANSIBLE VERSION
```

```

##### CONFIGURATION

##### OS / ENVIRONMENT

##### SUMMARY

##### STEPS TO REPRODUCE

```yaml
- hosts: "{{ hosts }}"
  gather_facts: no
  serial: 1
  tasks:

  ### raw with unicode arg and output

  - name: raw with unicode arg and output
    raw: echo 汉语
    register: command
  - name: check output of raw with unicode arg and output
    assert:
      that:
      - "'汉语' in command.stdout"
      - command | changed # as of 2.2, raw should default to changed: true for consistency w/ shell/command/script modules
```
If the above doesn't copy correctly you can find it in `ansible/test/integration/targets/connection/test_connection.yml`

Running with `ANSIBLE_USE_PERSISTENT_CONNECTIONS`
##### EXPECTED RESULTS

##### ACTUAL RESULTS
```
TASK [raw with unicode arg and output] ********************************************************************************************************************************************************************************************************
fatal: [local-pipelining]: FAILED! => {"changed": true, "failed": true, "rc": 255, "stderr": "Traceback (most recent call last):\n  File \"/home/johnb/git/ansible-inc/ansible/bin/ansible-connection\", line 184, in run\n    (rc, stdout, stderr) = self.conn.exec_command(cmd)\n  File \"/home/johnb/git/ansible-inc/ansible/lib/ansible/plugins/connection/local.py\", line 72, in exec_command\n    display.vvv(u\"EXEC {0}\".format(cmd), host=self._play_context.remote_addr)\nUnicodeDecodeError: 'ascii' codec can't decode byte 0xe6 in position 5: ordinal not in range(128)\n", "stdout": "", "stdout_lines": []}
```



Discussion:

Resolving patch:
diff --git a/lib/ansible/plugins/connection/local.py b/lib/ansible/plugins/connection/local.py
--- a/lib/ansible/plugins/connection/local.py
+++ b/lib/ansible/plugins/connection/local.py
@@ -28,7 +28,7 @@
 from ansible.compat import selectors
 from ansible.compat.six import text_type, binary_type
 from ansible.errors import AnsibleError, AnsibleFileNotFound
-from ansible.module_utils._text import to_bytes, to_native
+from ansible.module_utils._text import to_bytes, to_native, to_text
 from ansible.plugins.connection import ConnectionBase
 
 
@@ -67,7 +67,7 @@ def exec_command(self, cmd, in_data=None, sudoable=True):
 
         executable = C.DEFAULT_EXECUTABLE.split()[0] if C.DEFAULT_EXECUTABLE else None
 
-        display.vvv(u"EXEC {0}".format(cmd), host=self._play_context.remote_addr)
+        display.vvv(u"EXEC {0}".format(to_text(cmd)), host=self._play_context.remote_addr)
         display.debug("opening command with Popen()")
 
         if isinstance(cmd, (text_type, binary_type)):


Test patch:
Repository: ansible/ansible

Issue:
ios_command: ValueError: invalid keys: answer
##### ISSUE TYPE
 - Bug Report

##### COMPONENT NAME
ios_command

##### ANSIBLE VERSION
```

```


##### SUMMARY

Possible edge case/regression introduced by https://github.com/ansible/ansible/pull/21713/files

##### STEPS TO REPRODUCE
```
ansible -m ios_command -a 'commands=?' -u ec2-user -i ios-csr1000v, ios-csr1000v -e 'ansible_host=ec2-54-173-135-107.compute-1.amazonaws.com ansible_port=22 ansible_connection=local ansible_ssh_private_key_file=/root/.ansible/test/id_rsa'
```

##### EXPECTED RESULTS

##### ACTUAL RESULTS
```
2017-02-22 16:32:57  [WARNING]: Removed unexpected internal key in module return: _ansible_parsed =
2017-02-22 16:32:57 False
2017-02-22 16:32:57 
2017-02-22 16:32:57 An exception occurred during task execution. To see the full traceback, use -vvv. The error was: ValueError: invalid keys: answer
2017-02-22 16:32:57 ios-csr1000v | FAILED! => {
2017-02-22 16:32:57     "changed": false, 
2017-02-22 16:32:57     "failed": true, 
2017-02-22 16:32:57     "module_stderr": "Traceback (most recent call last):\n  File \"/tmp/ansible_iElgFY/ansible_module_ios_command.py\", line 232, in <module>\n    main()\n  File \"/tmp/ansible_iElgFY/ansible_module_ios_command.py\", line 201, in main\n    responses = run_commands(module, commands)\n  File \"/tmp/ansible_iElgFY/ansible_modlib.zip/ansible/module_utils/ios.py\", line 80, in run_commands\n  File \"/tmp/ansible_iElgFY/ansible_modlib.zip/ansible/module_utils/ios.py\", line 75, in to_commands\n  File \"/tmp/ansible_iElgFY/ansible_modlib.zip/ansible/module_utils/network_common.py\", line 150, in __call__\n  File \"/tmp/ansible_iElgFY/ansible_modlib.zip/ansible/module_utils/network_common.py\", line 107, in __call__\nValueError: invalid keys: answer\n", 
2017-02-22 16:32:57     "module_stdout": "", 
2017-02-22 16:32:57     "msg": "MODULE FAILURE", 
2017-02-22 16:32:57     "rc": 0
2017-02-22 16:32:57 }
```



Discussion:

Resolving patch:
diff --git a/lib/ansible/module_utils/ios.py b/lib/ansible/module_utils/ios.py
--- a/lib/ansible/module_utils/ios.py
+++ b/lib/ansible/module_utils/ios.py
@@ -69,7 +69,7 @@ def to_commands(module, commands):
     spec = {
         'command': dict(key=True),
         'prompt': dict(),
-        'response': dict()
+        'answer': dict()
     }
     transform = ComplexList(spec, module)
     return transform(commands)
diff --git a/lib/ansible/modules/network/ios/ios_command.py b/lib/ansible/modules/network/ios/ios_command.py
--- a/lib/ansible/modules/network/ios/ios_command.py
+++ b/lib/ansible/modules/network/ios/ios_command.py
@@ -168,7 +168,6 @@ def main():
     """main entry point for module execution
     """
     argument_spec = dict(
-        # { command: <str>, prompt: <str>, response: <str> }
         commands=dict(type='list', required=True),
 
         wait_for=dict(type='list', aliases=['waitfor']),


Test patch:
Repository: ansible/ansible

Issue:
S3 module: Make bucket deletion a more explicit command
##### ISSUE TYPE
 - Feature Idea

##### COMPONENT NAME
s3

##### ANSIBLE VERSION
```
ansible 2.2.1.0
  config file = 
  configured module search path = Default w/o overrides
```

##### CONFIGURATION
N/A

##### OS / ENVIRONMENT
Local: OSX
Remote: Ubuntu 14

##### SUMMARY
The ``s3`` module parameter ``mode=delete`` for deleting bucket is conceptually similar to ``mode=delobj``.  Prevent users from deleting buckets rather than objects by making ``object`` parameter and ``mode=delobj`` mutually exclusive in task.  Alternatively in the future replace ``mode=delete`` with a more explicit ``mode=delete_bucket``.  This is a particularly important distinction as S3 buckets are, by default, without versioning - the accidental removal of a bucket is something to be avoided.

##### STEPS TO REPRODUCE / CHANGES SUGGESTED

<!--- Paste example playbooks or commands between quotes below -->

1.  Make ``object`` parameter and ``mode=delobj`` mutually exclusive parameters
2.  Alias then replace ``mode=delete`` with ``mode=delete_bucket``  

```yaml
- name: Delete s3 bucket rather than object
  s3:
    bucket="{{ s3_bucket }}"
    object="{{ s3_path }}/file.tar.gz"
    mode=delete
    region="{{ region }}"

- name: Delete s3 object rather than bucket
  s3:
    bucket="{{ s3_bucket }}"
    object="{{ s3_path }}/file.tar.gz"
    mode=delobj
    region="{{ region }}"

# Proposed change
- name: Delete s3 bucket rather than object
  s3:
    bucket="{{ s3_bucket }}"
    mode=delete   # or alternatively "delete_bucket"
    region="{{ region }}"

- name: Delete s3 object rather than bucket
  s3:
    bucket="{{ s3_bucket }}"
    object="{{ s3_path }}/file.tar.gz"
    mode=delobj
    region="{{ region }}"
```

I would also suggest changing ``mode=delete`` to something more explicit like ``mode=delete_bucket`` in future, but at least preventing the object parameter might make things more explicit than it is now. 



Discussion:

Resolving patch:
diff --git a/lib/ansible/modules/cloud/amazon/s3.py b/lib/ansible/modules/cloud/amazon/s3.py
--- a/lib/ansible/modules/cloud/amazon/s3.py
+++ b/lib/ansible/modules/cloud/amazon/s3.py
@@ -511,6 +511,10 @@ def main():
     if module.params.get('object'):
         obj = module.params['object']
 
+    # Bucket deletion does not require obj.  Prevents ambiguity with delobj.
+    if obj and mode == "delete":
+        module.fail_json(msg='Parameter obj cannot be used with mode=delete')
+
     # allow eucarc environment variables to be used if ansible vars aren't set
     if not s3_url and 'S3_URL' in os.environ:
         s3_url = os.environ['S3_URL']


Test patch:
Repository: ansible/ansible

Issue:
nxos_facts: from ansible.module_utils.network_common import exec_command
##### ISSUE TYPE
 - Bug Report
##### COMPONENT NAME
nxos_facts

##### ANSIBLE VERSION
```
b8d308d919c89d86dc72d4fd62335b9f948544f1
```

##### SUMMARY
```

An exception occurred during task execution. The full traceback is:
Traceback (most recent call last):
  File "/tmp/ansible_PV6Oik/ansible_module_nxos_facts.py", line 184, in <module>
    from ansible.module_utils.network_common import exec_command
ImportError: cannot import name exec_command

fatal: [nxos01]: FAILED! => {
    "changed": false, 
    "failed": true, 
    "module_stderr": "Traceback (most recent call last):\n  File \"/tmp/ansible_PV6Oik/ansible_module_nxos_facts.py\", line 184, in <module>\n    from ansible.module_utils.network_common import exec_command\nImportError: cannot import name exec_command\n", 
    "module_stdout": "", 
    "msg": "MODULE FAILURE", 
    "rc": 0
}
	
```
##### STEPS TO REPRODUCE
`time ANSIBLE_ROLES_PATH=targets  ansible-playbook -vvvvv  -i  inventory.network-jobarker nxos.yaml  -e "limit_to=nxos_facts"`


##### ACTUAL RESULTS



Discussion:
cc @jedelman8 @GGabriele @privateip @Qalthos
[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: notify --->

Resolving patch:
diff --git a/lib/ansible/modules/network/nxos/nxos_facts.py b/lib/ansible/modules/network/nxos/nxos_facts.py
--- a/lib/ansible/modules/network/nxos/nxos_facts.py
+++ b/lib/ansible/modules/network/nxos/nxos_facts.py
@@ -181,7 +181,7 @@
 from ansible.module_utils.nxos import run_commands
 from ansible.module_utils.nxos import nxos_argument_spec, check_args
 from ansible.module_utils.basic import AnsibleModule
-from ansible.module_utils.network_common import exec_command
+from ansible.module_utils.connection import exec_command
 from ansible.module_utils.six import iteritems
 
 
diff --git a/lib/ansible/plugins/action/nxos.py b/lib/ansible/plugins/action/nxos.py
--- a/lib/ansible/plugins/action/nxos.py
+++ b/lib/ansible/plugins/action/nxos.py
@@ -99,7 +99,9 @@ def run(self, tmp=None, task_vars=None):
             self._task.args['provider'] = provider_arg
 
         # make sure a transport value is set in args
-        if self._task.args.get('transport') is None and self._task.args.get('provider').get('transport') is None:
+        transport = self._task.args.get('transport')
+        provider_transport = (self._task.args.get('provider') or {}).get('transport')
+        if all((transport is None, provider_transport is None)):
             self._task.args['transport'] = 'cli'
 
         return super(ActionModule, self).run(tmp, task_vars)


Test patch:
Repository: ansible/ansible

Issue:
nxos_nxapi: Not idempotent 
##### ISSUE TYPE
 - Bug Report
##### COMPONENT NAME
nxos_nxapi

##### ANSIBLE VERSION
```
b8d308d919c89d86dc72d4fd62335b9f948544f1
```

##### SUMMARY
Looks like nxapi isn't detecting the following or for some othe reason wants to send the following commands
```yaml
    "changed": true, 
    "commands": [
        "nxapi https port 9443"
    ], 
```


On a second run, and therefore isn't idempotent

##### STEPS TO REPRODUCE
`
time ANSIBLE_ROLES_PATH=targets  ansible-playbook -vvvvv  -i  inventory.network nxos.yaml  -e "limit_to=nxos_nxapi"
`


##### ACTUAL RESULTS



Discussion:
cc @jedelman8 @GGabriele @privateip @Qalthos
[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: notify --->

Resolving patch:
diff --git a/lib/ansible/modules/network/nxos/nxos_nxapi.py b/lib/ansible/modules/network/nxos/nxos_nxapi.py
--- a/lib/ansible/modules/network/nxos/nxos_nxapi.py
+++ b/lib/ansible/modules/network/nxos/nxos_nxapi.py
@@ -192,14 +192,14 @@ def map_obj_to_commands(updates, module):
 def parse_http(data):
     match = re.search('HTTP Port:\s+(\d+)', data, re.M)
     if match:
-        return {'http': True, 'http_port': match.group(1)}
+        return {'http': True, 'http_port': int(match.group(1))}
     else:
         return {'http': False, 'http_port': None}
 
 def parse_https(data):
     match = re.search('HTTPS Port:\s+(\d+)', data, re.M)
     if match:
-        return {'https': True, 'https_port': match.group(1)}
+        return {'https': True, 'https_port': int(match.group(1))}
     else:
         return {'https': False, 'https_port': None}
 


Test patch:
Repository: ansible/ansible

Issue:
nxos_config: Not idempotent when used with src
##### ISSUE TYPE
 - Bug Report
##### COMPONENT NAME
nxos_command

##### ANSIBLE VERSION
```
b8d308d919c89d86dc72d4fd62335b9f948544f1
```

##### SUMMARY
Running the following playbook shows a change

```yaml
- name: setup
  nxos_config:
    commands:
      - no description
      - no shutdown
    parents:
      - interface Ethernet2/5
    match: none
    provider: "{{ cli }}"

- name: configure device with config
  nxos_config:
    src: basic/config.j2
    provider: "{{ cli }}"
  register: result

- assert:
    that:
      - "result.changed == true"

- name: check device with config
  nxos_config:
    src: basic/config.j2
    provider: "{{ cli }}"
  register: result

- assert:
    that:
      - "result.changed == false"

```

##### STEPS TO REPRODUCE
```
cd test/integration
time ANSIBLE_ROLES_PATH=targets  ansible-playbook -vvvvv  -i  inventory.network-jobarker nxos.yaml  -e "limit_to=nxos_config"
```

Note that depending when you look at this bug you may have to comment out the `- "result.updates is not defined"` tests

##### EXPECTED RESULTS

##### ACTUAL RESULTS



Discussion:
cc @jedelman8 @GGabriele @privateip @Qalthos
[click here for bot help](https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md)
<!--- boilerplate: notify --->

Resolving patch:
diff --git a/lib/ansible/modules/network/nxos/nxos_config.py b/lib/ansible/modules/network/nxos/nxos_config.py
--- a/lib/ansible/modules/network/nxos/nxos_config.py
+++ b/lib/ansible/modules/network/nxos/nxos_config.py
@@ -235,7 +235,7 @@ def get_running_config(module):
         if module.params['defaults']:
             flags.append('all')
         contents = get_config(module, flags=flags)
-    return NetworkConfig(indent=1, contents=contents)
+    return NetworkConfig(indent=2, contents=contents)
 
 def get_candidate(module):
     candidate = NetworkConfig(indent=2)


Test patch:
diff --git a/test/integration/targets/nxos_config/tests/cli/src_basic.yaml b/test/integration/targets/nxos_config/tests/cli/src_basic.yaml
--- a/test/integration/targets/nxos_config/tests/cli/src_basic.yaml
+++ b/test/integration/targets/nxos_config/tests/cli/src_basic.yaml
@@ -14,6 +14,7 @@
 - name: configure device with config
   nxos_config:
     src: basic/config.j2
+    defaults: yes
     provider: "{{ cli }}"
   register: result
 
@@ -26,6 +27,7 @@
 - name: check device with config
   nxos_config:
     src: basic/config.j2
+    defaults: yes
     provider: "{{ cli }}"
   register: result
 
diff --git a/test/integration/targets/nxos_config/tests/cli/src_match_none.yaml b/test/integration/targets/nxos_config/tests/cli/src_match_none.yaml
--- a/test/integration/targets/nxos_config/tests/cli/src_match_none.yaml
+++ b/test/integration/targets/nxos_config/tests/cli/src_match_none.yaml
@@ -16,6 +16,7 @@
     src: basic/config.j2
     provider: "{{ cli }}"
     match: none
+    defaults: yes
   register: result
 
 - assert:
@@ -28,6 +29,7 @@
   nxos_config:
     src: basic/config.j2
     provider: "{{ cli }}"
+    defaults: yes
   register: result
 
 - assert:
Repository: ansible/ansible

Issue:
ansible fails as it tries to merge empty vars file
##### ISSUE TYPE
 - Bug Report

##### COMPONENT NAME
<!--- Name of the module/plugin/task/feature -->

##### ANSIBLE VERSION
```
ansible 2.2.1.0
```

##### CONFIGURATION
```
[ssh_connection]
pipelining=True
#ssh_args = -F ./ssh-bastion.conf -o ControlMaster=auto -o ControlPersist=30m
#control_path = ~/.ssh/ansible-%%r@%%h:%%p
[defaults]
host_key_checking=False
gathering = smart
fact_caching = jsonfile
fact_caching_connection = /tmp
stdout_callback = skippy
library = ./library
callback_whitelist = profile_tasks
```

##### OS / ENVIRONMENT
Ubuntu 16.04.2

##### SUMMARY
ansible is not able to merge 2 dicts when file is empty or commented.

##### STEPS TO REPRODUCE
```
$ ansible-playbook -i inventory/host.cfg book.yml  -c local  -e @inventory/group_vars/empty-all.yml
```
and
``` 
$ cat book.yml 
---
- hosts: localhost
  tasks:
    - name: debug msg
      debug: msg="Hi there"
```
and
```
$cat inventory/group_vars/empty-all.yml
# empty
```

##### EXPECTED RESULTS
have {} as dict if file is empty or everything is commented.

##### ACTUAL RESULTS
```
ERROR! failed to combine variables, expected dicts but got a 'dict' and a 'NoneType': 
{}
```


Discussion:

Resolving patch:
diff --git a/lib/ansible/utils/vars.py b/lib/ansible/utils/vars.py
--- a/lib/ansible/utils/vars.py
+++ b/lib/ansible/utils/vars.py
@@ -20,7 +20,6 @@
 __metaclass__ = type
 
 import ast
-import os
 import random
 import uuid
 
@@ -120,6 +119,7 @@ def merge_hash(a, b):
 def load_extra_vars(loader, options):
     extra_vars = {}
     for extra_vars_opt in options.extra_vars:
+        data = None
         extra_vars_opt = to_text(extra_vars_opt, errors='surrogate_or_strict')
         if extra_vars_opt.startswith(u"@"):
             # Argument is a YAML file (JSON is a subset of YAML)
@@ -130,7 +130,10 @@ def load_extra_vars(loader, options):
         else:
             # Arguments as Key-value
             data = parse_kv(extra_vars_opt)
-        extra_vars = combine_vars(extra_vars, data)
+
+        if data is not None:
+            extra_vars = combine_vars(extra_vars, data)
+
     return extra_vars
 
 


Test patch: