Repository: ansible/ansible

Issue:
python3 lookup csvfile error csvfile: argument 1 must be an iterator
<!---
Verify first that your issue/request is not already reported on GitHub.
Also test if the latest release, and devel branch are affected too.
-->

##### ISSUE TYPE
<!--- Pick one below and delete the rest -->
 - Bug Report

##### COMPONENT NAME
<!---
lookup csvfile
-->

##### ANSIBLE VERSION
<!--- Paste verbatim output from "ansible --version" between quotes below -->
```
ansible 2.4.3.0
 config file = /etc/ansible/ansible.cfg
 configured module search path = ['/root/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
 ansible python module location = /usr/local/lib/python3.5/dist-packages/ansible
 executable location = /usr/local/bin/ansible
 python version = 3.5.2 (default, Nov 23 2017, 16:37:01) [GCC 5.4.0 20160609]
```

##### CONFIGURATION
<!---
nothing changed, standard default configuration
-->

##### OS / ENVIRONMENT
<!---
Ubuntu 16.04 latest
-->

##### SUMMARY
lookup with csvfile failed with error message:
```
TASK [set_fact] ********************************************************************************************************
task path: /home/stevek/src/act2-ansible/tests/test-1.yml:8
Tuesday 27 February 2018  18:11:24 +1000 (0:00:00.691)       0:00:00.775 ****** 
fatal: [localhost]: FAILED! => {"msg": "An unhandled exception occurred while running the lookup plugin 'csvfile'. Error was a <class 'ansible.errors.AnsibleError'>, original message: csvfile: argument 1 must be an iterator"}
    to retry, use: --limit @/home/stevek/.ansible-retry/test-1.retry

PLAY RECAP *************************************************************************************************************
localhost    
```

##### STEPS TO REPRODUCE
<!---
For bugs, show exactly how to reproduce the problem, using a minimal test-case.
For new features, show how the feature would be used.
-->

<!--- Paste example playbooks or commands between quotes below -->

prepare the csv file /tmp/test.csv with content
```
ymbol,Atomic Number,Atomic Mass
H,1,1.008
He,2,4.0026
Li,3,6.94
Be,4,9.012
B,5,10.81
```

Playbook content:
```
- name: test
  hosts: localhost
  tasks:
    - set_fact:
        t: "{{ lookup('csvfile', 'Li file=/tmp/test.csv delimiter=,') }}"
       
```

Run ansible playbook and got the error

```
TASK [set_fact] ********************************************************************************************************
task path: /home/stevek/src/act2-ansible/tests/test-1.yml:8
Tuesday 27 February 2018  18:11:24 +1000 (0:00:00.691)       0:00:00.775 ****** 
fatal: [localhost]: FAILED! => {"msg": "An unhandled exception occurred while running the lookup plugin 'csvfile'. Error was a <class 'ansible.errors.AnsibleError'>, original message: csvfile: argument 1 must be an iterator"}
    to retry, use: --limit @/home/stevek/.ansible-retry/test-1.retry

PLAY RECAP *************************************************************************************************************
localhost    
```

Switch back to use python2 for ansible with exact ansible version and no error reported and it works as expected.

I switch by removing current ansible, and replace default pip with pip2 and install ansible using pip2.

<!--- You can also paste gist.github.com links for larger files -->

##### EXPECTED RESULTS
<!--- What did you expect to happen when running the steps above? -->

the lookup works

##### ACTUAL RESULTS
<!--- What actually happened? If possible run with extra verbosity (-vvvv) -->

Error describe above.


Discussion:

Resolving patch:
diff --git a/lib/ansible/plugins/lookup/csvfile.py b/lib/ansible/plugins/lookup/csvfile.py
--- a/lib/ansible/plugins/lookup/csvfile.py
+++ b/lib/ansible/plugins/lookup/csvfile.py
@@ -53,6 +53,7 @@
 
 from ansible.errors import AnsibleError, AnsibleAssertionError
 from ansible.plugins.lookup import LookupBase
+from ansible.module_utils.six import PY2
 from ansible.module_utils._text import to_bytes, to_native, to_text
 
 
@@ -66,8 +67,10 @@ def __init__(self, f, encoding='utf-8'):
     def __iter__(self):
         return self
 
-    def next(self):
-        return self.reader.next().encode("utf-8")
+    def __next__(self):
+        return next(self.reader).encode("utf-8")
+
+    next = __next__   # For Python 2
 
 
 class CSVReader:
@@ -77,13 +80,19 @@ class CSVReader:
     """
 
     def __init__(self, f, dialect=csv.excel, encoding='utf-8', **kwds):
-        f = CSVRecoder(f, encoding)
+        if PY2:
+            f = CSVRecoder(f, encoding)
+        else:
+            f = codecs.getreader(encoding)(f)
+
         self.reader = csv.reader(f, dialect=dialect, **kwds)
 
-    def next(self):
-        row = self.reader.next()
+    def __next__(self):
+        row = next(self.reader)
         return [to_text(s) for s in row]
 
+    next = __next__  # For Python 2
+
     def __iter__(self):
         return self
 
@@ -93,8 +102,8 @@ class LookupModule(LookupBase):
     def read_csv(self, filename, key, delimiter, encoding='utf-8', dflt=None, col=1):
 
         try:
-            f = open(filename, 'r')
-            creader = CSVReader(f, delimiter=to_bytes(delimiter), encoding=encoding)
+            f = open(filename, 'rb')
+            creader = CSVReader(f, delimiter=to_native(delimiter), encoding=encoding)
 
             for row in creader:
                 if row[0] == key:


Test patch: